Architecture
One Next.js process, one SQLite file, one disk volume. There is no worker container, no Redis and no external queue — the job runner is a singleton inside the web process, and every stage writes its artefact to disk before the job advances.
The moving parts
| Component | Lives in | Role |
|---|---|---|
| Route handlers | app/api/ | All Node-runtime, no Edge. Accept the upload, answer polls, serve downloads, open the progress stream. |
| Job runner | lib/runner/ | Module-level singleton. Claims queued rows into the prepare lane and prepared rows into synthesis, and reclaims lapsed leases on boot. |
| Extractors | lib/extract/ | One per format — PDF, EPUB, DOCX, ODT, RTF, HTML, Markdown, plain text. |
| Chunker | lib/text/ | Splits normalised text to a URL byte budget (CHUNK_MAX_URL_BYTES, 1800). |
| TTS client | lib/tts/client.ts | Calls synthesis.abair.ie under a p-limit semaphore. Sends PUBLIC_ORIGIN as Origin and Referer. |
| Video renderer | lib/video/ | Shells out to ffmpeg to burn the reading page and its captions into an MP4. |
| Database | drizzle/, lib/db/ | Drizzle over better-sqlite3. Migrations run on first open rather than at boot, so the image needs no TypeScript runtime; applied ones are recorded in __drizzle_migrations and skipped next time. |
A document's life
The pipeline runs in two lanes, with prepared as the seam:
queued → extracting → chunking → prepared → synthesising → stitching → ready
\___________ prepare lane __________/ \______ synthesis lane ______/
failed and cancelled are terminal. The prepare lane is ungated and has its own pool (RUNNER_PREPARE_SLOTS), so text exists within seconds of upload, long before the document's turn to synthesise; prepared is a resting state like queued, holding no lease.
Each chunk on disk is a resume point: a restart mid-document costs only the chunks in flight, because the runner reclaims rows left synthesising on boot and picks up from whichever WAVs exist. Rows whose watchdog lease lapses (RUNNER_LEASE_TTL_MS, five minutes) are re-queued the same way. Chunks go out one at a time by design — see Overview → Limits.
Editing the transcript does not discard the recording: lib/text/reconcile.ts matches stored chunk rows against a fresh chunking of the new text on exact equality, so untouched paragraphs keep their WAVs and only changed segments are re-synthesised. Chunk audio is named by row id for that reason — an edit renumbers reading order.
Storage
Everything lives under the data volume at /data.
| Path | Contents |
|---|---|
app.db | SQLite — jobs, chunks, renders |
uploads/ | The original uploaded file |
extracted/ | Normalised plain text |
chunks/ | Per-chunk WAVs — the resume points |
output/ | The stitched WAV |
renders/ | Rendered MP4s |
drizzle/migrations/meta/_journal.json must be committed. The migrator reads it to decide what to apply and throws Can't find meta/_journal.json file without it — the image builds cleanly and then fails on first database open, in production only.
Progress streaming
GET /api/jobs/[id]/events opens with a comment to flush proxy headers, replays whatever the in-memory bus has cached, then always sends a database-backed snapshot so a client never depends on the bus having survived a restart. A keep-alive ping every 15 seconds stops idle proxies reaping the socket, and X-Accel-Buffering: no stops nginx buffering the response. The vhost still needs proxy_http_version 1.1 — see Deployment → nginx.
Exports
| Export | Route | Notes |
|---|---|---|
| Stitched WAV | /api/jobs/[id]/audio | 16-bit mono 22050 Hz PCM |
| Extracted text | /api/jobs/[id]/text | Normalised plain text |
| Subtitles | /api/jobs/[id]/subtitles?format=srt|vtt | Derived on demand; no ffmpeg needed |
| Video | /api/jobs/[id]/renders → /api/renders/[id]/video | MP4 of the reading page, Range-aware |
Video is the only CPU-bound work in the container and shares a core with the web server, so ffmpeg (pinned at 8.0.1-r1) runs one render at a time niced to 15 — measured, nice 0 costs ~46% of web throughput against ~12% at nice 15. A font family is vendored at /app/assets/fonts (RENDER_FONTS_DIR), because Alpine ships none and libass renders nothing when it cannot resolve one while ffmpeg still exits 0.
Alpine builds ffmpeg with --enable-gpl --enable-version3, so the image redistributes a GPLv3 binary. The source offer is satisfied by pointing at Alpine's published sources.
The upload log
Everything about a document stays on the volume, with one exception: each accepted upload appends a metadata row to public.lei_uploads in the shared ABAIR auth Supabase project, so upload history outlives both the volume and the retention sweep.
The insert runs server-side under the user's own JWT, satisfying an "insert own rows" RLS policy — no service-role key, no extra environment variable — and is best-effort: an unreachable or un-migrated project logs supabase.upload_log_failed and the upload proceeds. The row carries filename, mime type, byte count, sha256, voice, speed and the owner's uid and email; never the document, its text, or its audio.
Failure modes worth knowing
| If this happens… | What users see |
|---|---|
synthesis.abair.ie is down | Jobs fail after their retries (RUNNER_AUTO_RETRY_MAX, 2). Finished documents still download. |
| The container restarts mid-document | The job resumes from the last completed chunk. |
| The runner dies without taking the process down | Pages load, nothing progresses. /api/health shows runner.running: false with queue.oldestQueuedAgeS climbing. |
| The disk fills | Uploads and synthesis fail. /api/health reports diskFreeBytes, so this is visible first. |
| ffmpeg is missing from the image | Video export is hidden, not offered and failed — /api/capabilities reports video.available: false. |
The lei_uploads table is missing | Nothing user-visible; a warning is logged. |
| The data volume is lost | Every document is gone. Only the metadata in lei_uploads survives. |
Drilling in
| Want to understand… | Read |
|---|---|
| How a job is claimed and driven | lib/runner/process.ts |
| Why a restart resumes instead of restarting | lib/runner/bootstrap.ts → lease reclaim |
| What is sent to the synthesis backend | lib/tts/client.ts → synth, getOrigin |
| Ownership scoping on every read | lib/db/repo.ts → getJobForUser, listJobsForUser |
| Session refresh and route gating | middleware.ts, lib/supabase/middleware.ts |
| Every tunable and its default | lib/config.ts |
Last updated 2026-09-16