PyData Amsterdam 2025

Sieves: Plug-and-Play NLP Pipelines With Zero-Shot Models
09-26, 14:55–15:30 (Europe/Amsterdam), Voyager

Generative models are dominating the spotlight lately - and rightly so. Their flexibility and zero-shot capabilities make it incredibly fast to prototype NLP applications. However, one-shotting complex NLP problems often isn't the best long-term strategy. Decomposing problems into modular, pipelined tasks leads to better debuggability, greater interpretability, and more reliable performance.

This modular pipeline approach pairs naturally with zero- and few-shot (ZFS) models, enabling rapid yet robust prototyping without requiring large datasets or fine-tuning. Crucially, many real-world applications need structured data outputs—not free-form text. Generative models often struggle to consistently produce structured results, which is why enforcing structured outputs is now a core feature across contemporary NLP tools (like Outlines, DSPy, LangChain, Ollama, vLLM, and others).

For engineers building NLP pipelines today, the landscape is fragmented. There’s no single standard for structured generation yet, and switching between tools can be costly and frustrating. The NLP tooling landscape lacks a flexible, model-agnostic solution that minimizes setup overhead, supports structured outputs, and accelerates iteration.

Introducing Sieves: a modular toolkit for building robust NLP document processing pipelines using ZFS models.

In NLP, implementing pipelines of established tasks is a common practice. Prototypes that require little to no data are often very valuable during early system development, as fully supervised models typically demand extensive annotated datasets. Generative and predictive zero-/few-shot (ZFS) learning models address this limitation by eliminating the immediate need for full annotation, enabling a “warm start” over a “cold start” with fully supervised models. This approach accelerates development and enables faster feedback loops.

Despite the richness of the open NLP ecosystem, there currently is no toolkit supporting rapid task prototyping with ZFS models without tight coupling to a specific (1) model, (2) structured generation library (e.g. https://github.com/stanfordnlp/dspy, https://github.com/dottxt-ai/outlines, https://github.com/1rgs/jsonformer, https://github.com/BoundaryML/baml) or (3) task. All three components should be as interchangeable as possible to improve on the developer experience and efficacy. sieves aims to fill this gap.

Agenda

This talk will cover the following topics:

  • The Value of Pipelines and Task Isolation: Exploring their usefulness as abstractions, even when using generative models.
  • Warm Starts With Prototyping: How to iterate effectively without relying on a training set.
  • Overview of Structured Generation Tools: A summary of the current landscape in structured generation.
  • Introducing Sieves: Bringing it all together with pipelines, warm starts/prototyping, and structured generation.

The largest part of the presentation will focus on sieves, its core concepts, and its benefits:

  • Loose coupling: Decoupling models, tasks, and structured generation backends for flexibility.
  • Unified I/O format: Standardized input/output formats across models, tasks, and structured generation backends.
  • Generative and predictive ZFS support: Compatibility with both approaches for versatility.
  • Document- and pipeline-based architecture: Designed for maintainability and easy debugging.
  • Comprehensive task support: Comes with plenty of task implementations for ZFS regimes, enabling single-line execution for tasks such as: vlassification, NER, entity linking, guided information extraction, REL, translation, summarization, etc.
  • Auxiliary component integration: Support for essential tools often needed in practical NLP workflows, e.g. for file parsing and chunking.
  • Customizability: Lightweight encapsulation of tasks, allowing users to easily customize or add new tasks.
  • Supervised training support: Enables compilation and export of training, validation, and test datasets in popular formats like Hugging Face’s datasets to simplify model distillation on LLM output.

Takeaways

By the end of this talk, attendees will understand:

  • The Benefits of Pipelines and Prototyping For Warm Starts: How pipelines and warm starts - leveraging prototypes with ZFS techniques - can speed up the development of NLP applications, also when using generative models.
  • The Role of Structured Generation in Building Real-World NLP applications: Insights into the current state of structured generation for text models and its impact on task implementation.
  • Contributions of Sieves: The place of sieve in the NLP ecosystem and how it contributes to a smoother and more productive developer experience in building out NLP applications by simplifying prototyping and accelerating development.