The Ai KhanLet's talk
← All case files

Case file 011Captions

The problem

“Captions take longer than the video. Can the captions make themselves?”

Solved

For
Creators posting talking-head Reels, Shorts and TikToks
Built
A web app: upload, transcribe, edit, render, download
Stack
Next.js · Drizzle · Deepgram / AssemblyAI / Whisper · ffmpeg + ASS subtitles
Result
Metric to confirm
Before: the video is done; the captions take all night3D pixel film · made by me

01 · The pain

The video took ten minutes. The captions take the evening.

  • Every word typed in by hand, and lined up with the audio by ear.
  • Every line break chosen one at a time, so the text doesn’t cover the face.
  • Urdu speech comes back in Urdu script, when the audience reads Roman Urdu.

02 · The fix

Upload a video. Pick a style. Get an MP4.

A sample clip with word timings, grouped by the tool’s real rules. Press play or drag the timeline, and switch styles without transcribing again.

0.0s/10.6s
The hardest part

Phrase grouping · grouping rules from the real tool

  • 2–5 words per phrase
  • 42 characters soft cap
  • 350ms pause forces a break
p1
The hardest part
The
0.0s–0.4shardest
0.5s–1.1spart
1.2s–1.8s
3 words16 chars1.8s duration
p2
of making viral videos
of
2.2s–2.5smaking
2.6s–3.3sviral
3.4s–4.0svideos
4.0s–4.1s
4 words22 chars1.9s duration
p3
is the captions.
is
4.5s–4.9sthe
5.0s–5.4scaptions.
5.5s–6.2s
3 words16 chars1.7s duration
p4
Nobody reads them
Nobody
6.8s–7.4sreads
7.5s–8.0sthem
8.1s–8.4s
3 words17 chars1.6s duration
p5
doing them by hand.
doing
8.9s–9.4sthem
9.5s–9.9sby
10.0s–10.3shand.
10.4s–10.6s
4 words19 chars1.7s duration

Sample

This lab uses real grouping rules from the Caption Tool’s code. The transcript above shows how words get automatically grouped into phrases. Try scrubbing the timeline: captions appear phrase-by-phrase, with each word highlighted as it “speaks.” Switch between styles to see how the same caption data renders in different looks–viral-yellow is small and clean, hormozi-white is big and bold.

In production, the tool transcribes your video, groups words into phrases using these rules, lets you edit and pick a style, then burns the captions into an MP4.

03 · How it works

Transcribe once, restyle as often as you like.

  1. Drop the clip in

    An MP4 or MOV up to 3 minutes. The length is checked in the browser, and the file uploads straight to storage.

  2. Every word, with its time

    Speech-to-text returns each word with its start and end. If one service fails or has no key, the next one takes over: Deepgram, then AssemblyAI, then Whisper, then Hugging Face.

  3. Phrases, a style, an MP4

    Words are grouped into short phrases, one of 10 styles is applied, and the editor lets you fix any word. The server then burns the captions into the video with ffmpeg.

The hard part

Roman Urdu captions that still land on the right word.

Whisper writes Urdu speech in Urdu script, but people here read captions in Roman Urdu. A normal transliteration can merge or split words, and then every timing after it slides out of sync.

So the tool converts one word at a time and insists on the same number of words back, in the same order. Each word keeps the timestamp it came with. An AI model does the spelling, because rules alone can’t guess Urdu’s unwritten vowels.

آپ کیسے ہیں→ 3 words, 3 timestamps

aap kaise hain → still 3 words, same 3 timestamps

04 · What changed

Captions stop being the slow part.

Time to caption a clip
—
Caption styles
10
Speech services it can fall back on
4

Blank: metric to confirm.