2025-12-09 –, Machine Learning & AI
LLMs are powerful but things can quickly go wrong when you’re building real apps. Prompts may fail, tool calls can break, and outputs often behave in unexpected ways. In this talk, we’ll look at real-world issues developers face when working with Python-based LLM pipelines and how to fix them. You’ll walk away with practical debugging tips and tools to help make your AI apps more stable and trustworthy.
This talk shares lessons learned from hands-on experience building LLM-powered applications using Python. We'll walk through the common problems that show up in the real world like broken toolchains, failing prompts, inconsistent responses, and unexpected agent behaviors.
The session focuses on practical debugging strategies: how to trace issues, log properly, test prompts like code, and use tools like langsmith or simple logs to spot problems early. We’ll also touch on memory management, retry logic, and why some issues only show up after your app goes live. Whether you're using LangChain, CrewAI, or your own pipeline logic, this talk will give you the mindset and techniques to debug faster, work smarter, and build more reliable AI systems.
No heavy theory just real engineering practices from the field.
Outline:
0–3 mins: Setting the Stage
3–8 mins: What Goes Wrong in Real Projects
8–14 mins: Shift Your Mindset, LLMs Are Code, Not Magic
14–20 mins: Debugging in Practice Tools That Help
20–26 mins: Lessons That Made My Life Easier
26–30 mins: Final Thoughts + Q&A
Key Takeaways
Know where and why LLM apps break in real workflows
Use open-source tools to debug prompts, agents, and memory
Apply versioning, logging, and testing strategies to prompt chains
Learn debugging practices that improve developer productivity
Yes
Kalyan is a Data and AI scientist with a background as a former data science and analytics manager, effectively balancing both academia and industry. He has presented talks at various PyCon's, Data Science and AI conferences, showcasing his expertise. As a community leader, Kalyan currently serves as the one of the Chair for PyConf Hyderabad 2025 and has held the role of Co-chair for PyCon India 2023 and PyConf Hyderabad from 2022 to 2024. In addition to these leadership positions, he is an active contributor to numerous Python, data science, and scientific communities worldwide.