06-08, 16:15–17:00 (Europe/London), Doddington Forum
Suppose you want to write a data science tool to do feature engineering. Your experience may go like this:
- Expectation: you can focus on state-of-the art techniques for feature engineering.
- Reality: you keep having to make you codebase more complex because a new dataframe library has come out and users are demanding support for it.
Or rather, it might have gone like that in the pre-Narwhals era. Because now, you can focus on solving the problems which your tool set out to do, and let Narwhals handle the subtle differences between different kinds of dataframe inputs!
Narwhals is a lightweight and extensible compatibility layer between dataframe libraries. It is already used by several open source libraries including Altair, Marimo, Plotly, Scikit-lego, Vegafusion, and more. You will learn how to use Narwhals to build dataframe-agnostic tools.
This is a technical talk aimed at tool-builders. You'll be expected to be familiar with Python and dataframes. We will cover:
- 2-3 minutes: Motivation. Why are there so many dataframe libraries?
- 2-3: minutes: Life before vs after Narwhals - real-world examples of how the data landscape is changing
- 7-8 minutes: Basics of Narwhals, wrapping native objects, expressions vs Series, lazy vs eager
- 7-8 minutes: Advanced Narwhals concepts: row order, non-elementary group-by aggregations, multi-indices, null values, backwards-compatibility promises
- 10 minutes: What is the Narwhals community like, how can you contribute and get involved, what comes next?
- 5-10 minutes: Engaging Q&A / awkward silence
Tool builders will benefit from the talk by learning how to build tools for modern dataframe libraries without sacrificing support for foundational classic libraries such as pandas.
Previous knowledge expected
Marco is the author of Narwhals, core contributor to pandas and Polars, and works at Quansight Labs as Senior Software Engineer. He also consults and trains clients professionally on Polars. He has also written the first Polars Plugins Tutorial and has taught Polars Plugins to clients.
He has a background in Mathematics and holds an MSc from the University of Oxford, and was one of the prize winners in the M6 Forecasting Competition (2nd place overall Q1).