Tag

#audio

July 28, 20265 min read

Emotion detection that must never slow the voice

In short: the emotion classifier in the supervised care companion runs beside the reply, never in front of it. It starts when the user's words arrive, nothing waits for it, and when it finishes it…

July 14, 20265 min read

Barge-in is the whole product

In short: a voice companion you cannot interrupt is a recording with extra steps. Barge-in, the user talking over the reply and the reply stopping, is what makes it feel like a conversation, and it…

November 4, 20255 min read

Streaming speech: where the latency actually goes

In short: in a voice pipeline that waits for each stage to finish, the first sound waits for the whole answer to be written and then spoken. Streaming doesn't make any stage faster. It lets the stages…

October 21, 20254 min read

A voice assistant in a weekend: LLM to speech over WebSockets

In short: a working voice assistant is one WebSocket, one language model call and one text-to-speech call, in that order. I built one in a weekend with FastAPI and OpenAI's APIs. It works, it's…

September 9, 20255 min read

Speech to facial animation: a week with Audio2Face

In short: NVIDIA's Audio2Face turns speech into a stream of ARKit blendshape weights, with emotion folded in. It gives a talking character believable lips with no animator, but it is one stage in a…

April 8, 20254 min read

Monitoring a model that listens to patients

In short: for a voice model in healthcare, the true labels arrive late or never, so you can't watch accuracy. You watch the recordings coming in, the scores going out, and the plumbing in between, and…

March 25, 20255 min read

Cloud Run or EC2: how I chose for a model that listens

In short: if the service takes a finished recording, scores it and answers, Cloud Run is usually the simpler and cheaper home. If it holds a live audio stream open, needs a GPU, or has to stay warm…

March 11, 20255 min read

Audio preprocessing that matters more than the model

In short: on medical audio, the steps before the model decide most of what the model can learn. Resampling, trimming, filtering, fixed-length windows and a split by patient each remove a shortcut the…

December 17, 20244 min read

Respiratory sound classification, end to end

In short: I took a public set of lung sound recordings all the way from raw audio to a web demo anyone can upload a file to. The model was the smallest part. Most of the work was deciding what to…

November 5, 20244 min read

Real-time transcription on a laptop: what Whisper gets wrong

In short: live on a laptop, Whisper invents text in silence, repeats itself, transcribes your speakers, cuts words at chunk edges and mangles names. Most of the fixes sit around the model, not in it:…