2025-09-24 –, Katherine Johnson @ TNW City
Retrieval-Augmented Generation (RAG) systems rely heavily on the quality of the retrieval process to generate accurate and contextually relevant outputs. In this 90-minute tutorial, we explore practical techniques to enhance retrieval across three key stages: pre-retrieval, mid-retrieval, and post-retrieval. Participants will learn how to optimize data preparation, query strategies, reranking, and evaluation to significantly improve the performance of RAG systems. A real-world case study will guide attendees through implementing these methods in a complete retrieval workflow.
Description
The tutorial provides a structured approach to optimizing retrieval in RAG pipelines through the Retrieval Playground - a comprehensive Python toolkit for RAG experimentation and evaluation. We'll cover:
* Pre-retrieval optimization: Document chunking strategies (baseline, recursive, unstructured, and docling), query enhancement techniques (expansion, decomposition, rewriting, self-querying), and semantic routing for domain-appropriate retrieval.
* Mid-retrieval optimization: Advanced retrieval methods including MMR for diversity, score thresholding, metadata filtering, cross-encoder reranking, and hybrid search.
* Post-retrieval optimization: Four document combination strategies, plus comprehensive RAGAS-based evaluation.
Participants will explore practical techniques through interactive Jupyter notebooks and hands-on implementations. We'll demonstrate the complete workflow through structured tutorials, showing how to implement and evaluate different techniques at each stage to build more efficient and intelligent retrieval pipelines.
Outline:
- Introduction to Retrieval in RAG Systems
- Hands-on Tutorial Walkthrough: Four Interactive Notebooks
- Pre-Retrieval Optimization
- Document Chunking
- Query Enhancement
- Semantic Routing
- Evaluation
- Mid-Retrieval Optimization
- Core Retrieval Methods
- Advanced Techniques
- Reranking
- Hybrid Approaches
- Post-Retrieval Optimization
- Document Chain Methods
- Implementation Approaches
- Method Selection
- Evaluation & Continuous Improvement
- RAGAS Metrics
- Performance Benchmarking
- Model Management
- Q&A and Wrap-up
Background Knowledge Required: - Familiarity with Python
- Basic understanding of LLMs
Target Audience: ML engineers, data scientists, developers working with LLMs in production, and anyone looking to learn how to build robust AI workflows using open source tools.
Materials:
Git Repo: https://github.com/mahimaarora/retrieval-playground/tree/main
Blogs:
https://medium.com/@mahimaarora025/next-level-retrieval-in-rag-pre-retrieval-strategies-part-1-5c967878e240
https://medium.com/@mahimaarora025/next-level-retrieval-in-rag-retrieval-strategies-part-2-9297604d15fb
https://medium.com/@mahimaarora025/next-level-retrieval-in-rag-post-retrieval-strategies-part-3-8ba504dd8223
https://medium.com/@mahimaarora025/building-trustworthy-ai-evaluation-metrics-for-rag-systems-explained-part-2-bd2bcff03021
Aarti Jha is a Senior Data Scientist at Red Hat, where she develops AI-driven solutions to streamline internal processes and reduce operational costs. She brings over 6.5 years of experience in building and deploying data science and machine learning solutions across industry domains. She has been an active part of the PyData community and presented at PyData NYC 2024.
Mahima Arora is a Senior Data Scientist on the Data & AI team at Red Hat, specializing in Generative AI applications. She develops AI-powered solutions that enhance efficiency and effectiveness, leading initiatives to optimize AI systems for greater impact. An open-source enthusiast, Mahima continuously explores new tools and technologies to expand her expertise and stay at the forefront of innovation.