2025-12-09 –, Planck-Bohr
Python is the language of choice for anything to do with AI and ML. While that has made it easy to write code for one machine, it's much more difficult to run workloads across clusters of thousands of nodes. Ray allows you to do just that. I'll demonstrate how to implement this open source tool with a few lines of code. As a demo project, I'll show how I built a RAG for the Wheel of Time series.
DESCRIPTION
I don’t read as much as I used to. As a result, ploughing through the Wheel of Time series by Robert Jordan has become a multi-year project for me. With 2787 named characters across 14 books, I often have to search on wikis for what happened six books ago. And who was that one minor character again…?
With AI we can do better. What if I fine-tuned an LLM with RAG to help me keep track of the entire saga? I could ask questions and it would give me spoiler-free answers… ✨
Whipping up a few lines of Python to train and tune models is easier than ever. But while executing scripts on a laptop or VM is trivial, it becomes much more difficult when you want to parallellize such a workload across an entire cluster. That’s where Ray comes in. With a few lines of code, we can modify our code to distribute our model training to tens or hundreds of machines. If you ever wondered how you companies like OpenAI train foundational models: this is how.
In this talk, I’ll cover Ray’s core concepts and show how to bring it in action. I’ll introduce Ray tasks and actors, and show how to set up large-scale pipelines with Ray Data. As a demo project, I’ll finetune an LLM to help me with my Wheel of Time read-through. And I’ll show how can scale your workloads from a single machine to thousands of nodes.
AGENDA
- Introduction (2m)
- Problem (3m)
- Ray explained (10m)
- Demo project (10m)
- Conclusion and takeaways (5m)
KEY TAKEAWAYS
- For AI workloads we need frameworks that are hardware-agnostic and work on both CPUs and GPUs
- With Ray tasks and actors, we can achieve distributed computing with just a few decorators
- Automatic up- and downscaling is a must if you want to keep your cloud bill in check
- While a literary masterpiece, The Wheel of Time has too many characters and some pacing issues
Rob de Wit-Liezenga is a freelance engineer who has worked in different data-related roles throughout his career. He's been a data analyst, data platform engineer, developer advocate, and customer success engineer. He enjoys work where he can combine hard tech with a people-focused role.
He has spoken at and co-organized various conferences in the past, including PyData Eindhoven. He enjoys meeting fellow pythonistas, learning from them, and sharing some of his own learnings.
Nowadays, Rob works as a CSE for Anyscale, where he helps customers deploy and optimize their infrastructure for large-scale AI workloads. Outside work, he enjoys photography, learning Spanish, hiking, and climbing.