The Ai KhanLet's talk
← All case files

Case file 014Voice input on Linux

The problem

“I have a lot to say, but typing it all is slow. Why can’t my computer just listen?”

Solved

For
Me first: dictation on a Linux (Wayland) desktop
Built
afk-voicy: a floating mic, a tray icon, a settings window and a CLI
Stack
Python · Whisper on Groq / Hugging Face · Deepgram · GTK4
Result
Metric to confirm
Before: a head full of words, and two fingers to type them3D pixel film · made by me

01 · The pain

Talking is faster than typing. Linux wasn’t listening.

  • Long messages typed slowly, when saying them would take seconds.
  • Dictation tools built for Windows and Mac, and silent on a Linux desktop.
  • Raw transcripts full of “um” and “uh”, pasted in and fixed by hand.

02 · The fix

Speak into a floating button. Watch it clean your words.

A sample transcript. Switch each stage on and off. The filler step uses the app’s real pattern; the word list and AI pass are simulated here.

Sample · Raw transcription from Whisper

Um, so I've been thinking about, uh, how to improve the, erm, deployment pipeline. We need to, um, make it faster. Ahm, specifically, the, uh, kubernetes cluster needs better, erm, resource allocation.

Strip “um”, “uh”, “erm”, “ahm” and collapse spaces — removed 7 fillers

So I've been thinking about, how to improve the, deployment pipeline. We need to, make it faster. specifically, the, kubernetes cluster needs better, resource allocation.

Replace custom terms (e.g. kubernetes → Kubernetes)

Punctuation, capitalisation, no false starts (simulated)

Final output

So I've been thinking about, how to improve the, deployment pipeline. We need to, make it faster. specifically, the, kubernetes cluster needs better, resource allocation.

03 · How it works

Mic to type, with streaming and reconciliation.

  1. Click the floating mic button or use a hotkey

    The orb shows the state: grey idle, red listening, amber transcribing. It stops on silence, or when you say “stop listening”.

  2. Speech becomes text while you talk

    Whisper (on Groq or Hugging Face) or Deepgram turns the audio into text, and partial results arrive while you are still speaking. Fillers like “um” and “uh” are stripped on the spot.

  3. Text is cleaned, then typed

    Your own word list fixes names and jargon. An optional AI pass tidies punctuation and false starts. Then the text is typed into whichever window has focus, by whichever method that window accepts.

The hard part

Never show corrections while you’re still talking.

Waiting for the full sentence feels slow, but typing every guess means words flicker and change on screen. So while you speak, a fresh transcript is requested every 1.2 seconds, and only the words that two transcripts in a row agree on are typed. The last two are always held back, because they still change.

When you stop, the final version is compared with what is already on screen, and only the rest is typed.

Partial 1

so I want to build

Partial 2

so I want to build a

Typed so far (2 held back)

so I want

Final

so I want to build a system

Action

type “to build a system”

04 · What changed

Say it once. It appears where the cursor is.

Words dictated
—
Time saved a day
—
People using it
—

Blanks: metrics to confirm.