Scaling Background Noise Filtration for AI Voice Agents
In the world of AI voice agents, especially in sensitive contexts like healthcare, audio clarity is everything. Background noise—a barking dog, a TV, street sounds—degrades transcription accuracy, leading to slower, clunkier, and less reliable AI responses. But how do you solve this in real-time without breaking the bank?
This talk chronicles our journey at a health-tech startup to ship background noise filtration at scale. We'll start with the core principles of noise reduction and our initial experiments with open-source models, then dive deep into the engineering architecture required to scale a compute-hungry ML service using Python and Kubernetes. You'll learn about the practical, operational considerations of deploying third-party models and, most importantly, how to measure their true impact on the product.
This is a "from the trenches" talk detailing the technical and strategic journey of shipping a real-time, compute-intensive ML feature at a resource-constrained AI startup. The core challenge was not just filtering background noise, but deploying this model into a reliable production environment using Python and Kubernetes.
More than just a scaling story, this talk is a case study in what it truly means to make a model-driven service "production ready." We'll cover the full lifecycle: from initial experiments with open-source models like DeepFilterNet, to the trade-offs that led us to a commercial SDK (Krisp), and finally to the architectural decisions required to operate it at scale. We will also tackle a question every data team faces: how do you know your compute-hogging workload is actually making the product better?
The session is structured as a practical journey and is designed to be accessible to a broad PyData audience of engineers, data scientists, and technical leaders.
Stephen Cheng is a software engineer at Parakeet Health, an AI powered voice agent startup that serves medical providers, where he works on infrastructure and backend. He has also worked at Uber and Microsoft.