Storage Architecture

S3-compatible object storage (MinIO in dev/CI, any S3 in production) as the single file store, reached only through presigned URLs minted by the API. Sibling docs: system · backend · database · search · security.

Pipeline in one line: client → API (auth + plan limits + key minting) → presigned PUT to MinIO → completion (HEAD + magic-byte sniff) → SCANNING → worker (scan / metadata / thumbs, TASK-0102) → READY.

---

0. Phase 9 Implementation Notes (recorded decisions)

• One pair of buckets per environment, all workspaces shared — S3_BUCKET_FILES (private uploads) + S3_BUCKET_THUMBS (read-only serving, still server-authorized). Tenant isolation is by key prefix (files/{workspaceId}/…) plus server-side permission checks — not by bucket ACL, per the documented TASK-0101 Requirement 2 decision. Both buckets are Private ACL; there is no public-read bucket and no bucket-policy public prefix. The raw object key is never returned to clients (the FileRecord shape omits key); clients only ever receive short-lived signed URLs.
• SigV4 canonical-header set. Presigned requests must sign exactly the headers that are NOT passed as query params: host (always) and any extra signed headers (content-type for PUT). x-amz-date travels as the X-Amz-Date query param in presigned form and is therefore never in X-Amz-SignedHeaders (reference-verified against boto3: presigned PUT signs content-type;host, presigned GET signs host only). Header-signed (CRUD) requests via S3Client require host in the signed set too — signS3Request injects it. Getting this wrong produces 400 "headers present which were not signed" / 403 SignatureDoesNotMatch on every S3 round trip.
• Upload response headers are exactly { "Content-Type": <mime> }. The spec's Content-Length is intentionally not returned (recorded deviation, TASK-0101): (1) the presigner does not content-length-lock (the hand-rolled signer does not support the content-length signed header, so the spec's "when the S3 SDK supports it" is "does not"); (2) echoing the declared size as a Content-Length header makes strict HTTP clients (undici, and several curl/SDK code paths) refuse to send a PUT whose body is shorter than declared — which is exactly the under-PUT attack the 422 UPLOAD_INCOMPLETE completion check exists to catch, so the test suite must be able to make that PUT and observe 422 server-side. The declared size is still enforced at completion by HEAD-ing the object and comparing Content-Length (422 on mismatch).
• No duplicate content-type header. Returning both the API's Content-Type and the signer's lowercase content-type yields a duplicated header on the PUT; undici joins the values (image/png,image/png) and MinIO answers 403 SignatureDoesNotMatch (the signed value is the single mime). The deduplicated single Content-Type is the fix.
• Workspace resolution for uploads. Requests carry x-factory-workspace; without it the guard falls back to claims.workspaces[0] (the JWT's first membership), which is the wrong workspace for a user who owns several. The plan-limit and quota checks (effectiveMaxFileSize / effectiveStorageQuota) are therefore keyed on the header-named workspace; tests bind to a workspace explicitly.
• Bucket bootstrap. scripts/minio-init.ts (pnpm storage:bootstrap) is idempotent: authoritative existence check via GET / (ListAllMyBuckets), create-if-missing, then verifies privacy by asserting an anonymous GET on the bucket yields 403. (Earlier version mis-read anonymous HEAD 403s on a missing bucket as "exists" and never created them — repaired.)

TASK-0104 (versions / trash / permissions / quota) recorded decisions

• A version is a re-upload, not a new object on a new endpoint. POST /files/uploads { replaceFileId } re-points the EXISTING File row at a new files/{ws}/{fileId}/{n+1}/… key; the worker finalizes a FileVersion n+1. Old version objects are retained in the bucket (their FileVersion rows stay) until POST /files/:id/versions/prune (keeps the current, deletes the rest) or a purge. Rollback (POST /files/:id/rollback {version}) is a key swap — File.key/size re-point at the target version's existing object; no copy, no re-scan.
• The raw object key never leaves the server. This includes the version list (GET /files/:id/versions returns version/size/created/current, no key) — the only content path is a per-read signed URL (§5). (The original versions() implementation leaked key: v.key; fixed and unit-pinned.)
• Storage quota is an EVENT counter, not a live sum. Workspace.usageStorageBytes (BigInt) is updated transactionally with the byte-moving operation: complete +size, prune −Σ, purge −Σ, restore ±0 (bytes retained while trashed). The pre-check reads it O(1); only the DENIAL boundary re-checks the authoritative ledger sum, and only a stale-LOW counter (drift > 1 GB) is corrected before a 413. A naive Σ File.size/Σ FileVersion.size recompute undercounts retained VIRUS_DETECTED/FAILED bytes (no FileVersion row) and misses in-flight SCANNING bytes — so it is reconciler-only (recomputeWorkspaceStorageBytes / authoritativeStoredBytes in modules/files/quota.ts), never the live path.
• Delete is a soft trash; purge is the hard delete. DELETE /files/:id sets deletedAt (object + rows retained; search doc removed via the file.deleted event); POST /:id/restore clears it (re-emits file.uploaded {restored:true}); DELETE /:id?purge=true deletes every version object + the whole thumbnail set + the rows in one transaction (FULL_ACCESS on the file, or workspace Admin/Owner). The thumbnails are removed by a deletePrefix sweep of thumbs/{ws}/{fileId}/ (modules/files/s3.ts), not by the single File.thumbnailKey (which records only the 480 primary variant) — a 1400px PNG owns 160/480/1280 artifacts, so deleting only the recorded key would orphan the other two (bug caught by the TASK-0104 lifecycle integration spec).
• File permissions: most-specific wins, NONE is a hard deny. The effective level is the max of the role baseline and the actor's explicit grants on the most-specific NAMED resource (file > record > page > database

  team > workspace); an explicit NONE grant on any named scope in that chain (the file itself or its page) overrides everything else → 403 + file.download_denied audit. Implemented in apps/api/src/lib/authorize.ts (see the §conflict note in docs/tasks/phase-09-files.md: the spec named packages/permissions, but the live permission walk is authorize.ts — the API does not depend on @factory/permissions; only the realtime app does).

• Orphan sweep. Objects under files/ with no matching File/ FileVersion row (crashed finalizes, partial transactions) are reclaimed by apps/worker/src/processors/files/cleanup.ts::runOrphanSweep — the daily maintenance:orphans repeatable (7-day grace on the object's Last-Modified) and scripts/storage-orphans.ts (pnpm storage:orphans, dry-run by default; --purge deletes; --file <id> scopes) share the core. Each deletion is audited file.orphan_purged. The core imports only @prisma/client + @factory/storage (no worker logger) so the ROOT script can import it (the root has no pino hoist).

1. Key layout

files/{workspaceId}/{fileId}/{version}/{safeName}     ← files bucket (private)
thumbs/{workspaceId}/{fileId}/{size}/{stem}.{ext}     ← thumbs bucket (private, serving)

• size ∈ 160 | 480 | 1280 (px wide; THUMBNAIL_SIZES).
• safeName = sanitizeFilename(name): basename across both / and \ (traversal is impossible by construction — ../../etc/passwd → passwd), control chars + <>:"\/*?| → _, total ≤ 180 chars with the extension preserved case-insensitively (UPPER.PNG keeps .PNG), collision suffix -<8 hex> when two files in the same workspace+file slot share a name.
• Keys are server-built from fileId + sanitized name; the client name never reaches a key path unsanitized. parseFileKey / parseThumbKey are the strict inverses (throw on anything malformed).
• Determinism: no clock, no randomness in key construction (only the caller- supplied collision hex), so re-minting a key is stable.

2. Plan limits (source of truth)

packages/shared/src/constants.ts::PLAN_LIMITS + apps/api/src/modules/files/uploads.service.ts::effectiveMaxFileSize and apps/api/src/modules/files/quota.ts::PLAN_USAGE_LIMITS / effectiveStorageQuota (the plan-limit + storage-quota min-rule; moved to quota.ts in TASK-0104 so the workspace usage endpoint shares the exact same ceiling — uploads.service re-exports them):

plan	maxFileSize	storageQuota
free	50 MB (52 428 800)	1 GB (1 073 741 824)
paid	500 MB (524 288 000)	10 GB (10 737 418 240)

• Per-file: effectiveMaxFileSize = min(plan.maxFileSize, usageLimits.maxFileSize (if > 0), LIMITS.MAX_FILE_SIZE=500 MB). Exceeded → 413 FILE_TOO_LARGE with the limit in the body.
• Workspace quota: effectiveStorageQuota = min(plan.storageQuota, usageLimits.storageQuota) — no hard ceiling (admin may raise above plan). size > quota − usageStorageBytes → 413 QUOTA_EXCEEDED (naming used + limit). The running usageStorageBytes counter is an event counter maintained by TASK-0104 (see the §0 TASK-0104 decisions) — column + backfill migration 20260929000000_workspace_usage_storage_bytes; the pre-check reads it O(1) and only the denial boundary re-checks the ledger sum.
• Workspace.usageLimits (Json, default {}) is the per-workspace override; an empty object means "pure plan defaults".

3. Upload session lifecycle

POST /api/v1/files/uploads          → 201 { upload: { file, uploadUrl, method:"PUT",
                                                    headers:{"Content-Type":mime}, expiresAt } }
   client PUT <uploadUrl> <bytes>   (signed 900 s, content-type-locked, key-bound)
PUT  /files/uploads/:id/complete    → 200 SCANNING (HEAD exists + size matches + sniff)
PUT  /files/uploads/:id/complete    → 422 UPLOAD_INCOMPLETE (size mismatch, never READY)
PUT  /files/uploads/:id/complete    → 410 UPLOAD_EXPIRED   (object gone: expired / never PUT)
PUT  /files/uploads/:id/complete    → 415 MIME_MISMATCH    (sniff ≠ claim; object DELETED, row FAILED)
POST /files/uploads/:id/cancel      → 200 { deleted:true } (S3 object + row removed)

• Session creation: auth + EDIT on the target page/block (or workspace root when unattached), size + quota pre-checks (§2), allow-list check on the claimed MIME, File row (status UPLOADING), signed PUT.
• Completion: HEAD the object (404 → 410 UPLOAD_EXPIRED), size compare (422), download first 8 KiB → sniffMagicBytes (415 MIME_MISMATCH → object deleted + row FAILED + scanReport.result="mismatch"), then status=SCANNING + enqueue files job process-file { fileId, version: 1 }.
• S3_SIGN_EXPIRY_SECONDS=900 (15 min) for both PUT and GET; the signed expiry is asserted live by the integration suite (test G shortens it to 2 s and exercises the identical server path).

3a. File processing pipeline (worker, TASK-0102)

SCANNING hands off to the worker, which owns every transition out of it. The pipeline is a sequential, timed step machine (apps/worker/src/processors/files/processor.ts → runFilesPipeline), each step tagged file:{fileId}:step:{name} for observability:

download → scan → metadata → thumbnails → finalize
   │         │        │          │           └─ READY | VIRUS_DETECTED | FAILED
   │         └─ clamav verdict  └─ dims/media/pdf  └─ planThumbnails render
   └─ streamed to a temp file (sha256 in-pass)

• Download — the object is streamed to a private temp file (S3Client.downloadTo), computing the sha256 in the same pass (a 500 MB object is never held in RAM). A disk-space guard (free-space statfs) refuses a run that would not fit. The temp file is removed in a finally (a 5-min cron backstops a SIGKILL mid-run).
• Scan — ClamAV over the LOCAL temp copy (the object is never executed or parsed by anything but the metadata parsers): CLAMD_SOCKET → the unix socket zCONTSCAN protocol (what clamdscan uses, verified by strace); CLAMAV_HOST → TCP zINSTREAM; neither → dev mode (WARN + skip, scanReport { engine:"none", skipped:true }). A scan timeout / transport failure is RETRYABLE (BullMQ backoff), never a silent pass. Production fail-closed: NODE_ENV=production with no scanner refuses to boot.
• Metadata — the worker RE-detects the MIME from bytes (final authority — the client claim is never trusted) then extracts per kind: images → sharp dimensions + orientation; video/audio → ffprobe duration + codec; PDF → pdfjs page count. Extraction failures are warnings, not fatal (the file still becomes READY; only a corrupt object that cannot be THUMBNAILLED is fatal).
• Thumbnails — @factory/storage::planThumbnails (the pure plan) → render each artifact (sharp for images, pdfjs+canvas for PDF page 1, ffmpeg first frame for video) → PUT to the factory-thumbs bucket. File.thumbnailKey is the 480 variant. A decode failure is FATAL corrupt_object (the kind was identified by magic bytes, so a retry cannot fix it).
• Finalize — persists the terminal state + the event contract:
  • READY: status + final metadata (sha256/dimensions/media/pdf + pipeline bookkeeping) + scanReport + thumbnailKey; FileVersion upsert (idempotent — the API never creates the row, the worker owns it); file.uploaded exactly once (only on a fresh SCANNING→READY transition) + file.finalized (the search re-upsert seam); audit file.processed.
  • VIRUS_DETECTED: status + scanReport { engine:"clamav", version, infected, signature }; file.virus_detected event + workspace-admin notification; audit. NO thumbnails, NO file.uploaded, NO search update (the object stays stored for forensics; download is 403 per TASK-0103/0104).
  • FAILED: status + metadata.pipeline.error (+ code); audit file.process_failed.

Status machine (enforced; the full table is unit-tested): UPLOADING → SCANNING → READY | VIRUS_DETECTED | FAILED; READY → FAILED (re-scan only); VIRUS_DETECTED / FAILED terminal. Re-running a terminal file is a no-op.

Failure policy (errors.classifyError): RETRYABLE → rethrow so BullMQ backs off (S3 5xx, network/fetch, redis blips, clamd timeout); FATAL → mark the row FAILED and complete the job without a retry (S3 404 = upload_not_found, corrupt object, disk guard).

Proven by tests/integration/files-pipeline.integration.test.ts (real stack: Postgres/Redis/MinIO/ClamAV) + the worker unit suite (status table, error matrix, temp lifecycle, clamd protocol, boot gate).

4. MIME detection (packages/storage/src/mime.ts)

Magic-byte sniffing is authoritative over the client claim:

1. MIME_SIGNATURES leading-byte table (order = specificity): PNG, JPEG, GIF, WebP (RIFF…WEBP @8), the ISO-BMFF family disambiguated by the ftyp BRAND @8 (avif / heic(mif1) / m4a(M4A ) / mov(qt ) / mp4(ftyp @4)), PDF, WebM (EBML), OGG, MP3 (ID3 + frame sync), WAV, ZIP.
2. zip-container refinement for OOXML (docx/xlsx/pptx/epub) — a bare zip magic + container sniff yields the refined type.
3. Extension fallback (~60 ext → MIME), then a text heuristic. Each result carries confidence: magic | ext | text | unknown.
4. Allow-list policy (isAllowedUpload) runs on the claimed type at session creation; the worker re-detects from bytes after the object lands (untrusted client input is never trusted).
5. SVG: allowed but served attachment-only (Content-Disposition: attachment
   • X-Content-Type-Options: nosniff) — the stored-XSS vector is closed at the storage layer, not just the UI.

5. Signed GET + preview

• GET /api/v1/files/{id}/download → 302 to a signed GET (response-content-disposition: attachment; filename="…", 900 s).
• GET /api/v1/files/{id}/preview → 302 signed GET (inline for image/video/audio/pdf; attachment otherwise, e.g. SVG).
• The URL signs exactly one object key: the canonical request includes the object resource, so reusing the signature against a different key 403s (key-binding — proven in the integration suite by swapping the key in the path of a valid signature). After 900 s S3 returns RequestExpired.
• Both routes require ≥ VIEW on the file (TASK-0104 defines file-permission resolution; here: ≥ VIEW on the containing page/block or an explicit file grant). Non-member → 403 FORBIDDEN + file.download_denied audit.

6. Serving model

There is no public URL for any object. Every byte transfer is either (a) a presigned PUT/GET the API minted (browser talks to S3 directly for the upload/download body) or (b) a 302 from the API to such a URL. The API's server-side S3 credentials never leave the server. The thumbs bucket is "read-only serving" in the sense that the pipeline is the only writer and reads are signed — not in the sense that it is publicly readable.

7. Security invariants (proven by tests/integration/files.integration.test.ts)

• Private buckets: unsigned/anonymous GET on a real object key → 403 in BOTH buckets (test H).
• Wrong Content-Type on the presigned PUT → S3 403 (signature covers the type) → completion 410 UPLOAD_EXPIRED; the object never lands (test C).
• Under-PUT (body < declared size) → completion 422 UPLOAD_INCOMPLETE; the row never reaches READY (test B).
• Text bytes claimed image/png → 415 MIME_MISMATCH, object deleted from the bucket (asserted via storage-client listing), row FAILED (test D).
• Signed-URL expiry (900 s contract; 2 s in test) → S3 403 → 410 UPLOAD_EXPIRED (test G). Key-binding swap → 403 (test F). Tampered signature → 403 (test F). Non-member download → 403 + audit (test F).
• Traversal: …/etc/passwd claim sanitizes to …/1/passwd (test I).
• Plan boundary: 60 MB → 413 free / 201 paid (test A).

8. Bucket provisioning

scripts/minio-init.ts (invoked by pnpm storage:bootstrap and the Phase-0 docker post-start): GET / ListAllMyBuckets → create missing buckets → verify private (anonymous GET → 403). Idempotent; safe to re-run. MinIO itself is a host process in dev (S3 on :9000, console :9001); production targets any S3-compatible endpoint via S3_ENDPOINT.