2025-11-08 –, Talk Track 2
Large language models are often too large to run on personal machines, requiring specialized hardware with massive memory. Quantization provides a way to shrink models, speed them up, and reduce memory usage - all while retaining most of their accuracy.
This talk introduces the fundamentals of neural network quantization, key techniques, and demonstrates how to apply them using Keras’s extensible quantization framework.
This talk introduces neural network quantization, its motivation, and how Keras makes it accessible through its built-in quantization tools. The session is designed to be both conceptual and hands-on: attendees will learn what quantization is and how to apply it directly in their workflows.
Target Audience: Software Engineers, Data scientists, ML engineers, and researchers interested in running models locally, or experimenting with model compression.
Background Knowledge: A basic understanding of neural networks (layers, forward pass, parameters) is sufficient. No advanced math is required.
The talk will be roughly divided into 3 parts:
Part 1 (0-10 min): Introduction to Quantization
We begin by understanding why model size is a barrier. We take an overview of quantization, where it helps, and how it democratizes AI for the GPU-middle class.
Part 2 (10-20 min): Fundamentals Explained Simply
We move on to understand the different kinds of quantization, some core math concepts, and tradeoffs between speed, accuracy, and memory.
Part 3 (20-35 mins): Quantization in Keras
Finally, we take a look at the Keras Quantization API, and quantize a real model. We end the session by taking a look at some performance and accuracy outcomes.
No previous knowledge expected
I'm a software engineer working on model optimization techniques in the Keras team at Google. I spend my time writing code in OSS, publishing new issues of my newsletter, or making YouTube videos!