avvy-worker runs long-running 3D model processing jobs that don't belong in the request path: preview rendering, metadata extraction, projection generation, and projection-derived 3D model synthesis. It dequeues asynq tasks from Redis, executes the pipeline, and POSTs HMAC-signed CBOR callbacks back to state-server.
Repo: gitlab.avvyland.com/avvy/avvy-worker.
state-server ── enqueue ──▶ Redis ── dequeue ──▶ avvy-worker ── pipeline ──▶ signed CBOR callback ──▶ state-server
Callback delivery is part of the job. A failed POST (network, TLS, 5xx) raises an asynq error and the task is retried — so all state-server callback handlers MUST be idempotent.
The single binary (cmd/avvy-worker) handles four task families. Each has its own callback signature header so the receiver can dispatch and verify.
| Task type | Produces | Callback signature header |
|---|---|---|
model.preview.v1 |
preview images + metadata | X-Model-Preview-Signature |
model.object_pipeline.v1 |
previews + metadata + processed CBOR + canonical model artifact set | X-Model-Object-Signature |
model.projection.step.v1 |
source / preprocessed / generated image + attempts and scores | X-Model-Projection-Signature |
model.projection.model.v1 |
generated 3D-model artifacts + previews | X-Model-Projection-Model-Signature |
Signature is HMAC-SHA256(raw_body_bytes, callback_secret) hex-encoded.
All task payloads and callbacks are CBOR — no JSON anywhere on the worker's interfaces.
Task payload (decoded fields):
model_hash (string) — stable content hash from state-server.input_format (string) — .glb / .gltf or recognized extension.input_model (bytes) — model bytes as a CBOR byte string.callback_url (string, required) — no env-var fallback; must be in the payload.Callback payload (common fields):
model_hash (string), task_id (string), status (completed | failed), error (string).metadata.{title, description, tags, estimated_dimensions, estimated_scale_factor, restricted_detected}.previews[] — angle / mime / width / height / image bytes.processed_cbor.{kind, logical_name, content_type, sha256, bytes} — public view.cbor artifact.canonical_model.{kind, logical_name, content_type, sha256, bytes} — internal canonical GLB.artifacts[], manifest — auxiliary assets and prebuilt manifests.Projection-step callbacks add job_id, step_id, sequence, prompt_original, prompt_used, the three image stages, and an attempts[] log.
Preview generation. Playwright + headless Chromium running a Three.js scene; configurable to install browsers on first run or use a pre-provisioned image.
Provider integrations (eight, verified in cmd/avvy-worker/main.go). OpenAI (OpenAIAPIKey), KIE (KieAPIKey), PIAPI (PiAPIKey), Gemini (GeminiAPIKey), NanoBanana (NanoBananaAPIKey), FAL (FalAPIKey), Replicate (ReplicateAPIKey), Tencent (TencentSecretID + TencentSecretKey). Each is gated on its key being non-empty; provider selection logic chooses among them per-task.
Metadata / image understanding. OpenAI, KIE, Gemini, plus PIAPI / NanoBanana / FAL where applicable; provider order is configurable.
Projection generation. Multi-provider with failover; deterministic prompt packs; per-attempt scoring against a configurable pass score (MODEL_PROJECTION_PASS_SCORE, default 0.8).
Projection 3D model. Tencent Hunyuan 3D 3.1 submit/poll → artifact download → canonical GLB normalization → textured preview render.
Background removal. Replicate (REPLICATE_BG_REMOVE_VERSION). The upscale option (REPLICATE_UPSCALE_VERSION) is deprecated and ignored by the projection pipeline.
Artifact storage. S3 (AWS SDK v2). Production target is Hetzner Object Storage (S3_ENDPOINT=https://fsn1.your-objectstorage.com, bucket avvyland, region fsn1, path-style) per avvy/argocd/avvy-worker/deployment.yaml.
Mesh ops. meshoptimizer vendored under meshopt/third_party/meshoptimizer (CGO; MIT-licensed).
Single YAML (config.yaml) with top-level keys: worker, previews, image_understanding, textures.
Worker load order: defaults → config.yaml (or MODEL_PREVIEW_WORKER_CONFIG path) → environment-variable overrides.
Pipeline load order: defaults → config.yaml (or BLENDERPROC_CONFIG).
In production, point both MODEL_PREVIEW_WORKER_CONFIG and BLENDERPROC_CONFIG at the same file unless intentionally splitting.
Environment variables follow these families (concrete values live in the repo's README; they are not mirrored here):
REDIS_ADDRESS, REDIS_PASSWORD, REDIS_DB.MODEL_PREVIEW_QUEUE_NAME, MODEL_PROJECTION_QUEUE_NAME, MODEL_PREVIEW_WORKER_CONCURRENCY, MODEL_PREVIEW_CALLBACK_SECRET (required), MODEL_PREVIEW_CALLBACK_TIMEOUT.OPENAI_*, KIE_*, GEMINI_*, REPLICATE_*, TENCENT_* family (region, endpoint, polling, concurrency limits).S3_BUCKET, S3_REGION, S3_ENDPOINT, S3_FORCE_PATH_STYLE.MODEL_PROJECTION_MAX_ATTEMPTS, MODEL_PROJECTION_PASS_SCORE.TRACING_ENABLED, OTLP_*, plus GRAFANA_OTLP_* convenience vars.OpenTelemetry → any OTLP endpoint (Grafana Tempo direct, or via Alloy / OTel Collector). Trace context propagates through asynq payloads, outbound provider calls, and the callback POST. When tracing is enabled, log lines include trace_id and span_id for major stages. See Observability.
go build -o bin/avvy-worker ./cmd/avvy-worker
go test ./...
CGO must be enabled (meshoptimizer). Runtime needs Redis, network egress to state-server callback URLs, the system TLS trust store, and Playwright Chromium.
A reference systemd unit and EnvironmentFile= example live in the worker's README.
Redis is trusted. An attacker who can enqueue tasks can choose callback_url and force the worker to POST model bytes anywhere. Keep Redis on a private network.
Rotate callback_secret as a shared secret — update worker and state-server together.
Scale horizontally by adding worker replicas on the same Redis queue. Keep concurrency low — preview rendering is CPU/memory-heavy.
Large models currently travel through Redis as CBOR bytes. A future revision will upload input to object storage and pass a URL in the payload.
Reads queues from Redis populated by state-server. Writes signed callbacks to state-server's callback endpoints. Stores artifacts in S3 owned by Storage.
gitlab.avvyland.com/avvy/avvy-worker — README.md, docs/projection-model-signoff-sop.md, cmd/avvy-worker/main.go (provider verification), cmd/avvy-worker/projection_task.go + projection_model_task.go (signature header constants). Deployment: gitlab.avvyland.com/avvy/argocd/avvy-worker/deployment.yaml.