One Job, One VM: The Engineering Behind AI Captions

We recently shipped AI captions. The announcement post covers what it does; this one covers the part I find more interesting — the architecture mistake we made in the first internal version, and what fixing it looked like.

#The mistake: captions as a second job

The first working version of captions treated transcription as its own job. Upload a video with captions enabled, and two things happened: a transcode job encoded your ladder, and a separate captions job transcribed the audio. Two jobs, two virtual machines, two downloads of the same source file.

It worked. It was also wrong in every dimension we care about:

• Cost. The second VM billed a full instance for work the first VM could have absorbed. Worse, the captions job re-downloaded the entire source — for a large mezzanine file, the redundant transfer could cost more than the compute.
• Speed. The captions job was scheduled independently, so it queued behind fleet capacity, booted its own machine, and downloaded the source again before the model saw a single audio sample. Captions routinely finished long after the video was playable.
• Sizing. A captions-only job has no video output, so our instance selector — which sizes machines by codec and resolution — had nothing to key on and fell back to defaults. Transcription ran on the smallest machine in the fleet, which is exactly where you don’t want to run a speech model.
• Legibility. Two jobs meant the cost of one user action was split across two records. The video page showed the encode cost and silently omitted the captions charge.

None of these were bugs in any single component. Every service did its job correctly. The topology was the bug.

#The fix: transcription is a phase, not a job

The correct model was hiding in plain sight: our worker already executes a job as a sequence of phases — download, probe, encode, package, upload. Transcription is just another phase. The source file is already on local disk. The machine is already paid for. The job record already exists to carry the fee.

So captions now run inside the encode job, as a phase on the same VM:

download → probe → encode → transcribe → package → upload

One subtlety earned its own design discussion: where in the sequence does transcription go? Our first instinct — transcribe before encoding, so a doomed transcription fails fast — is right for caption-only jobs and wrong for everything else. Speech-to-text on a long file takes real time, and putting it before the encode would delay the moment your video becomes playable by the entire transcription duration. Nobody should wait on a language model to watch their video.

So the ordering is conditional. Caption-only jobs (yes, you can run those — transcription with no encode) keep the fail-fast early slot. Jobs with video outputs transcribe after the encode completes but before packaging, which is the last moment the caption track can still be woven into the HLS/DASH manifests as a proper subtitle rendition. Players discover in-manifest captions natively; a sidecar file bolted on afterwards, they don’t.

The payoff, measured on a real captioned upload: total wall-clock dropped, the redundant source download disappeared entirely, and transcription now runs on the machine sized for your encode instead of the smallest box in the fleet — which happens to make the model meaningfully faster too.

#The billing guards

Folding captions into the encode job fixed the economics. Two smaller changes fixed the ethics.

An empty caption file must never bill. The speech model’s VTT writer emits a valid file header before it writes any cues — which means a recording with no detectable speech produces a well-formed, nine-byte, completely empty caption file. Every naive size check passes. Our first version would have uploaded that file, called it a success, and charged the captions fee. Now, zero surviving cues fails caption generation explicitly: the job reports it and the captions fee is voided — a caption-only job on a silent screencast fails outright and bills €0, and a captioned encode pays for the encode it delivered and nothing for the captions it didn’t.

Quality filtering must not gut the track. We filter hallucinated and low-confidence segments before delivery. Filters have failure modes: on difficult audio — heavy music beds, cinematic mixes — an aggressive confidence threshold can drop nearly everything and leave a “clean” track of three orphaned lines. Our rule now: if filtering would remove more than half the transcript, we ship the honest unfiltered transcription instead and say so. A complete track you can edit beats a sparse one that lies about the video.

Both guards follow the same principle we apply to encoding: the system should fail loudly rather than succeed dishonestly. A charge should always map to something delivered.

#Verified against real infrastructure, not mocks

The reorder touched the most load-bearing code path we have — the executor’s phase sequence — so unit tests weren’t enough. Before this reached production, our validation harness drove real caption jobs through real workers on the new build and asserted the things customers actually depend on: the produced media probes correctly, the caption rendition appears in the parsed HLS manifest, and the encode demonstrably completes before transcription begins. The same suite that backs our public conformance page gated this release like it gates every other.

The deployment order mattered too, and is worth stating because it generalizes: when an API change and a worker change ship together, the fleet must accept the new behavior before the API starts requesting it. We deployed the reordered worker first, verified it, and only then enabled the API-side flag. Contract producers last.

#Where this lands

A captioned upload is now one job, on one machine, with one visible cost, producing a video whose manifest carries its own captions. The architecture that was four kinds of wrong is now boring — which is the highest compliment infrastructure can earn.

There’s more coming behind the same door: bulk retro-captioning for existing libraries, caption webhooks, and timing work on cue boundaries. But the foundation — captions as a phase of the job you’re already paying for — is the part that had to be right first.