Introducing AI Captions

Captions are one of those features everyone agrees videos should have and almost nobody ships. They help viewers in loud rooms and quiet offices, they are how deaf and hard-of-hearing audiences watch at all, they feed search and SEO, and with the European Accessibility Act now in force they have moved from nice-to-have to obligation for a lot of products.

The reason they don’t get shipped is friction. Transcription is its own pipeline: extract audio, run a speech model somewhere, convert the output to WebVTT, store it next to the renditions, and get a player to actually discover it. Each step is easy; the chain is where projects die.

So we collapsed the chain. AI captions are now built into Transcodely — for hosted videos and for the raw transcoding API.

#One flag, if you’re on hosting

If you use Transcodely Video Hosting, captions are a boolean:

const { video, uploadUrl } = await client.videos.createUpload({
  appId: 'app_k1l2m3n4o5',
  filename: 'keynote.mp4',
  contentType: 'video/mp4',
  sizeBytes: 132710400,
  autoCaptions: true
});

That’s it. The same job that encodes your ladder also transcribes the audio, and the finished video comes back with a caption track attached: visible in the dashboard player, downloadable as WebVTT, and — for streaming outputs — referenced inside the HLS manifest itself, so any standards-compliant player discovers it with zero integration work.

#One line, if you bring your own storage

On the jobs API, caption generation is a subtitle operation, sitting next to the passthrough, convert, and extract operations you may already use:

const job = await client.jobs.create({
  inputUrl: 'https://your-bucket.example.com/keynote.mp4',
  outputOriginId: 'ori_x9y8z7w6v5',
  outputs: [{
    type: OutputFormat.HLS,
    video: [
      { codec: VideoCodec.H264, resolution: Resolution.RESOLUTION_1080P, quality: QualityTier.STANDARD },
      { codec: VideoCodec.H264, resolution: Resolution.RESOLUTION_720P, quality: QualityTier.STANDARD },
      { codec: VideoCodec.H264, resolution: Resolution.RESOLUTION_480P, quality: QualityTier.STANDARD }
    ],
    subtitleTracks: [
      { operation: SubtitleOperation.GENERATE, language: 'auto' }
    ]
  }]
});

The generated track lands in your bucket alongside the renditions — a .vtt (or .srt if you prefer), plus a segment-level transcript JSON if you want to build chapters, search, or clip tooling on top. Language can be pinned to a specific ISO 639-2 code or left on auto for detection.

#Priced like a utility, not like a seat

AI captions cost €0.05 per minute of source video. Flat, additive, and metered on the source — not per rendition. The audio is transcribed once per job whether you produce one output or a twelve-variant ladder, so the fee never multiplies with your encoding settings. It shows up as its own line item on the job, separate from encode cost — and separate from the flat per-job processing fee, which pays for starting an encoder and does not absorb a separately metered service.

Two honesty rules we committed to from day one:

• Silence is free. If the model finds no speech — silent screencasts, music-only content — the job reports it plainly and the captions fee is voided. You are never billed for an empty file.
• We don’t ship shredded tracks. Our quality filtering removes hallucinated and low-confidence segments, but if filtering would gut the majority of a transcript, we deliver the honest unfiltered transcription instead of a track with three orphaned lines. A bad-but-complete track you can edit beats a “clean” track that’s missing the video.

#Your audio never leaves our pipeline

This matters more than it sounds: there is no third-party AI API in this feature. Transcription runs on the same worker VM that encodes your video, using an open speech model (Whisper large-v3-turbo) that we operate ourselves. Your media is not sent to an external transcription vendor, doesn’t transit another company’s queue, and isn’t retained by anyone’s model-improvement program. The same isolation story you already get for encoding now covers speech-to-text.

Running it on the encode worker has a second benefit: the source is already there. No second download, no second machine, no waiting for a transcription queue — captions arrive with the transcode, on the same bill, attached to the same job.

#What’s in the box today

• auto_captions on hosted uploads (CreateUpload, CreateFromUrl, multipart)
• operation: "generate" subtitle tracks on API jobs — WebVTT or SRT output
• Automatic language detection, or a pinned language
• In-manifest caption renditions for HLS/DASH outputs
• Segment-level transcript JSON alongside the caption file
• Dashboard player support, caption download, and per-video cost visibility
• Available in all SDKs (Go, JavaScript, Python) and via our MCP server — your agents can caption videos too

#What’s next

Captions are becoming a first-class resource, not just a job artifact: retro-captioning your existing back catalogue in bulk, caption webhooks, editing and re-upload of tracks, and quality work on timing (cues that overstay silences are on our list — we notice these things too). If you have a captions workflow we should support, tell us.

Enable it on your next upload and see the track appear. It’s the least-effort accessibility win your videos will ever get.