How I got macOS dictation under 100ms with Deepgram

Murmur started as a simple idea: a menubar app that listens and types what you say. The first prototype used AVAudioRecorder to write audio to a temp file, then sent the file to a transcription REST API when you stopped speaking. It worked, but the latency was 2–4 seconds. That's too slow to feel like dictation — it feels like a batch job.

Streaming fixes this by sending audio as it's captured and receiving partial transcripts in real time. The latency drops to 70–110ms on a decent network connection.

Why file-based transcription is slow

There are two sources of latency in the naive approach. First, you have to wait until the user stops speaking to know the utterance is complete — you can't send a file you're still writing. Second, the REST round trip adds 500ms–2s depending on the model and server load.

Streaming eliminates both. You send audio chunks as they're captured. Deepgram returns partial transcripts as it processes incoming audio, and a final transcript when it detects end-of-speech.

Tapping the microphone with AVAudioEngine

AVAudioEngine lets you install a tap on the input node that fires a callback with a buffer of audio samples at a specified size. I use 1024 samples at 16kHz — that's about 64ms of audio per callback, which is a reasonable chunk size for streaming.

let engine = AVAudioEngine()
let inputNode = engine.inputNode
let format = AVAudioFormat(
    commonFormat: .pcmFormatInt16,
    sampleRate: 16000,
    channels: 1,
    interleaved: false
)!

inputNode.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in
    self.sendBuffer(buffer)
}

try engine.start()

The tap callback gives you a AVAudioPCMBuffer. You need to convert it to raw bytes to send over WebSocket.

func sendBuffer(_ buffer: AVAudioPCMBuffer) {
    guard let channelData = buffer.int16ChannelData else { return }
    let frameLength = Int(buffer.frameLength)
    let data = Data(bytes: channelData[0], count: frameLength * 2) // 2 bytes per int16
    webSocketTask?.send(.data(data)) { _ in }
}

Opening the WebSocket

Deepgram's streaming endpoint accepts the audio format in query parameters. You don't need to send a header or preamble — just open the connection and start sending bytes.

var components = URLComponents(string: "wss://api.deepgram.com/v1/listen")!
components.queryItems = [
    URLQueryItem(name: "encoding", value: "linear16"),
    URLQueryItem(name: "sample_rate", value: "16000"),
    URLQueryItem(name: "channels", value: "1"),
    URLQueryItem(name: "model", value: "nova-2"),
    URLQueryItem(name: "language", value: "en-IN"), // Indian English accent
    URLQueryItem(name: "interim_results", value: "true"),
]

var request = URLRequest(url: components.url!)
request.addValue("Token \(apiKey)", forHTTPHeaderField: "Authorization")

let session = URLSession(configuration: .default)
webSocketTask = session.webSocketTask(with: request)
webSocketTask?.resume()
receiveMessages()

Handling partial vs final transcripts

Deepgram sends two kinds of responses. Interim results have is_final: false — the transcript for audio it's processed so far, subject to change. Final results have is_final: true and speech_final: true — Deepgram detected a speech endpoint and committed to this transcript.

struct DeepgramResponse: Decodable {
    let isFinal: Bool
    let speechFinal: Bool
    let channel: Channel

    struct Channel: Decodable {
        let alternatives: [Alternative]
    }
    struct Alternative: Decodable {
        let transcript: String
    }
}

func handleMessage(_ data: Data) {
    guard let response = try? JSONDecoder().decode(DeepgramResponse.self, from: data) else { return }
    let transcript = response.channel.alternatives.first?.transcript ?? ""

    if response.speechFinal {
        // Commit this text — user finished a phrase
        insertText(transcript)
    } else if response.isFinal {
        // Update the in-progress display
        updatePreview(transcript)
    }
}

For Murmur, interim results update a floating preview. speech_final triggers the actual text insertion via CGEvent to the frontmost app.

Reconnect handling

WebSocket connections drop. The tap keeps running, the queue fills, and you lose audio. When the task fails, replace it and resume:

func receiveMessages() {
    webSocketTask?.receive { [weak self] result in
        switch result {
        case .success(let message):
            if case .data(let data) = message { self?.handleMessage(data) }
            self?.receiveMessages() // keep receiving
        case .failure:
            self?.reconnect()
        }
    }
}

Latency breakdown

On a 50Mbps home connection: tap buffer accumulation (~64ms) + WebSocket send + Deepgram inference + response = 70–110ms from speech to transcript. The dominant factor is Deepgram's inference time, which varies by model and load. nova-2 is fast enough that it's rarely the bottleneck.

The old file-based approach was 2–4 seconds. Streaming is about 30x faster for the first response.