Mohit Singh Chauhan
I am Senior Data Scientist at CVS Health and works in the Responsible AI and LLM/Agentic systems. My expertise lies in the technical aspects of ethical AI, with a particular focus on bias and fairness testing. I am dedicated to identifying and mitigating biases in AI systems to ensure they are fair and equitable for all users. Additionally, I specialize in hallucination detection and mitigation for large language models (LLMs), multi-modal models, and AI agents, striving to enhance the reliability and trustworthiness of these advanced technologies. The recent cutting-edge tools includes open-source libraries like LangFair and UQLM.
Session
As LLMs become increasingly embedded in critical applications across healthcare, legal, and financial domains, their tendency to generate plausible-sounding but false information poses significant risks. This talk introduces UQLM, an open-source Python package for uncertainty-aware generation that flags likely hallucinations without requiring ground truth data. UQLM computes response-level confidence scores from token probabilities, consistency across sampled responses, LLM judges, and tunable ensembles. Attendees will learn practical strategies for implementing hallucination detection in production systems and leave with code examples they can immediately apply to improve the reliability of their LLM-powered applications. No prior uncertainty quantification background required.