Image compression

Image upload in the wrapper is always asynchronous. The API never runs Sharp itself, so a CPU-bound compression can never block the event loop and starve request handling. An upload returns 202 Accepted immediately; a separate worker process does the actual compression and replaces the object in place; the client polls a job endpoint for the result.

The flow at a glance

See it in the demo

The bundled demo app drives this exact pipeline. A 9.0 KB PNG uploaded with the medium preset returns 202 immediately, the worker compresses it to a 966 B WEBP (a 90% saving), and the page polls the job until it shows the result:

Step 1 — Upload

curl -X POST "$BASE/project/users/MASTER_UUID/images" \
  -H "Authorization: Bearer $TOKEN" \
  -F image=@photo.png \
  -F preset=medium \
  -F format=webp \
  -F quality=80
# → 202 { "job_id": "…", "file_id": "…", "status": "pending" }

The endpoint (apps/api/src/routes/images.ts) requires the files:write scope and pins the subject. It:

1. Validates the image multipart field and the compression params.
2. Enforces the same plan constraints as a normal upload — max_file_size, allowed_mimes, storage_limit — plus the per-space sub-limit if the caller is a space token. Quota is charged against the original bytes (conservative: the worker frees the delta once it shrinks the object, so the system never over-admits).
3. Uploads the original to R2 and, in a single transaction, inserts the files row, increments storage_used, and inserts a pending compression_jobs row. One transaction means a mid-sequence failure never leaves a committed file row with no job, or a phantom storage charge.
4. Wakes the worker (best-effort Redis push) and returns 202.

An external-id variant exists: POST /project/users/ext/:externalId/images.

Parameters

All optional; a bare upload with just image defaults to webp @ q80, no resize.

Field	Values	Notes
preset	thumbnail | medium | large | original	a named bundle of format/quality/dimensions
format	webp | jpeg | png | avif	overrides the preset’s format
quality	1–100	overrides the preset’s quality
width	1–10000	max width (aspect preserved, never upscaled)
height	1–10000	max height (aspect preserved, never upscaled)
visibility	public | private	default private
folder, tags, classifier, metadata	—	same semantics as a normal file

Explicit parameters override the preset. The preset table (apps/api/src/lib/image-compression.ts):

Preset	Format	Quality	Width	Height
thumbnail	webp	75	150	150
medium	webp	80	800	—
large	webp	85	1920	—
original	webp	90	—	—

Resizing uses fit: 'inside' with withoutEnlargement: a dimension is a maximum bound, aspect ratio is preserved, and a small source is never upscaled. EXIF orientation is normalized so reported dimensions are the visually-correct ones.

Step 2 — The worker

The worker (apps/api/src/workers/image-compression.ts) is a separate process — the worker service in Compose, with no HTTP surface. Launch it with bun run dev:worker.

• Queue = Postgres (source of truth). claimNextJob() selects the oldest pending job with FOR UPDATE SKIP LOCKED, so any number of workers can run without ever double-claiming. Claiming increments the job’s attempts.
• Redis = wakeup only. awaitJob blocks on a Redis list (BRPOP) for low-latency wakeups. With no Redis configured, the loop degrades to polling — it still works, just less promptly.
• Compress with Sharp, then replace the object at the same storage key. Sharp output is deterministic for a given input + options, so a duplicate run (a reaper race) writes identical bytes — the replace is idempotent.
• Idempotent completion. The done transition is UPDATE … SET status='done' WHERE id=? AND status='processing'. If the reaper requeued the job and a second worker also ran it, exactly one wins the update; the loser skips the storage reconcile so the quota never double-decrements.
• Storage reconcile (winner only). files.size and mimeType are updated to the compressed values, and storage_used on both the user and (if any) the space is adjusted by the signed delta original − compressed via GREATEST(0, used − delta) — which both frees space when the image shrank and reclaims it in the rare case it grew.
• Result captured in the job’s result jsonb: originalSize, compressedSize, compressionRatio, format, width, height.

Step 3 — Poll the job

curl "$BASE/project/jobs/JOB_ID" -H "Authorization: Bearer $TOKEN"

// while pending / processing:
{ "job_id": "…", "file_id": "…", "status": "processing" }

// on success:
{ "job_id": "…", "file_id": "…", "status": "done",
  "original_size": 2400000, "compressed_size": 180000,
  "compression_ratio": 0.075, "format": "webp", "width": 1920, "height": 1080 }

// on failure (the original object stays usable, just uncompressed):
{ "job_id": "…", "file_id": "…", "status": "failed", "error": "…" }

GET /project/jobs/:jobId requires files:read. Ownership is checked in the handler (the path carries no :userId, so the path-based subject guard can’t apply): a subject-bound token sees only jobs for files it owns, and a mismatch returns 404, not 403, so job existence is never leaked across tenants or subjects.

Safety: the decompression-bomb guard

The upload route bounds the input byte size (the plan’s max_file_size), but a tiny, highly compressible file can still decode into a huge pixel buffer — a 16000×16000 solid PNG is ~256 MP ≈ ~1 GB of raw pixels — and OOM or wedge the single shared worker. That would be a cross-tenant denial of service.

compressImage caps two things (apps/api/src/lib/image-compression.ts):

• limitInputPixels — decoded pixel count capped at 50 MP (DEFAULT_MAX_INPUT_PIXELS), well below Sharp’s ~268 MP default.
• .timeout({ seconds }) — each pipeline is wall-clocked at 30 s (DEFAULT_COMPRESS_TIMEOUT_SEC).

A hostile image fails fast and cleanly — the job goes to failed and the original object is left untouched — instead of crashing the process and being requeued forever.

Crash recovery: attempts & dead-lettering

• A job left in processing by a killed worker is recovered by reapStaleJobs, which runs on boot and periodically. If the stale job has not exhausted its attempts it is returned to pending; if it has, it is dead-lettered (→ failed).
• MAX_ATTEMPTS = 3. A clean compressImage throw already fails the job on the first attempt; the attempt counter only backstops a hard process crash (a poison pill) so it cannot loop forever.

Known follow-ups

• Per-tenant fairness — a per-project in-flight cap (MAX_INFLIGHT_PER_PROJECT, currently 2) is enforced when claiming jobs: a tenant that floods the queue can occupy at most that many worker-pool slots, so other tenants always make progress. Each job is also bounded to ~30 s by the compression timeout.