Jonathan Shi
Jonathan is a software engineer on Snowflake's Snowpark Python team, and is a maintainer of the Modin project. He enjoys building systems that are usable, maintainable, and performant.
Session
12-09
17:30
30min
Bridging Interactive Data Science and Big Data with Hybrid Execution
Jonathan Shi
Hybrid Execution is a new capability introduced in the open-source Modin library that lets developers write familiar pandas code while automatically selecting the most efficient execution backend. Small datasets run locally for fast, interactive development, while larger workloads are transparently pushed down to distributed backends for scalable, high-performance execution. This approach enables faster development for rapid prototyping and iteration and future-proofs pipelines as data volumes grow.
General Track
General Track