2025-09-02 –, Kuppelsaal
Ever tried passing a Polars Dataframe to a data science library and found that it...just works? No errors, no panics, no noticeable overhead, just...results? This is becoming increasingly common in 2025, yet only 2 years ago, it was mostly unheard of. So, what changed? A large part of the answer is: Narwhals.
Narwhals is a lightweight compatibility layer between dataframe libraries which lets your code work seamlessly across Polars, pandas, PySpark, DuckDB, and more! And it's not just a theoretical possibility: with ~30 million monthly downloads and set as a required dependency of Altair, Bokeh, Marimo, Plotly, Shiny, and more, it's clear that it's reshaping the data science landscape. By the end of the talk, you'll understand why writing generic dataframe code was such a headache (and why it isn't anymore), how Narwhals works and how its community operates, and how you can use it in your projects today. The talk will be technical yet accessible and light-hearted.
Narwhals is a lightweight and extensible compatibility layer between dataframe libraries. It is already used by several major open source libraries including Altair, Bokeh, Marimo, Plotly, and more. You will learn how to use Narwhals to build dataframe-agnostic tools, how Narwhals gained traction in a short amount of time, and what the future of dataframes looks like.
This is a technical talk, and basic familiarity with Python and dataframes will be assumed. We will cover:
- What the data science landscape looked like in 2024 before Narwhals came onto the scene.
- What problems Narwhals solves, why you can't "just convert to pandas" or "just use PyArrow".
- How to use Narwhals, with an emphasis on lazy-only computation.
- Static typing.
- Narwhals and SQL.
- Extending Narwhals with your own backend.
- The Narwhals community, and how you can get involved.
- What we think the future of dataframes looks like, and how you can help make it happen.
Tool builders will learn how to build tools for modern dataframe libraries without sacrificing support for foundational classic libraries such as pandas. Data scientists will learn about what goes on under the hood when their favourite tools support their favourite dataframe libraries. Finally, everyone will learn from insights on community building and management.
Novice
Prerequisites:None
Abstract as a tweet (X) or toot (Mastodon):Do you use pandas or Polars every day? Want to see what’s next for DataFrame interoperability?
Marco is the author of Narwhals, core contributor to pandas and Polars, and works at Quansight Labs as Senior Software Engineer. He also consults and trains clients professionally on Polars. He has also written the first Polars Plugins Tutorial and has taught Polars Plugins to clients.
He has a background in Mathematics and holds an MSc from the University of Oxford, and was one of the prize winners in the M6 Forecasting Competition (2nd place overall Q1).