2025-11-05 –, Green Language: English
Production ML failures often stem from one overlooked issue: features that work perfectly in a notebook break during inference. This is the "notebook chaos"—the silent killer of ML systems. This session dives deep into the root cause: the translation gap between research and production. We'll explore the architectural pattern of a feature store, demonstrating how to eliminate training-serving skew, ensure point-in-time correctness, and build reliable pipelines that turn notebook ideas into production gold.
A model’s performance in a notebook is a promise; production is where that promise is often broken. This is "Notebook Chaos": the silent killer of ML systems where real-world predictions fail because features diverge between training and serving.
This session follows the story of Sarah, a data scientist deploying a churn prediction model. We'll diagnose why her high-performing notebook model fails in production, identifying the two root causes: feature translation drift and late-arriving data.
Having established the problem, we’ll introduce the solution: an open-source feature store (Feast). We will explore the core concepts that bridge the gap from research to production, including:
- "A Layer, Not a Platform": How a feature store integrates with your existing data stack (like Snowflake and Redis) without vendor lock-in.
- Point-in-Time Correctness: How to automatically prevent data leakage for training.
- Online/Offline Stores: A clear-eyed look at managing data for both training and low-latency inference.
You will leave with a clear, practical blueprint for implementing a production-grade feature management system.
I lead the AI/ML Engineering team at Voyantis, translating between data scientists, engineers, and business folks. My philosophy? Find the simplest solution that actually delivers.
I write about this stuff for Towards Data Science and practice yoga when I'm not debugging pipelines.