2025-11-05 –, AI
Tabbers is a suite of tools that transforms guitar-playing videos into rich, instructional content.
We use audio and video analysis to generate tablature, color-coded finger visualizations, and additional enrichments that would make it accessible and easy to learn how to play.
This is a great tool for you whether you’re a beginner trying to understand phrasing, an advanced player analyzing technique, or a tutor trying to teach a specific phrase.
Tabbers aims to reduce the friction between inspiration and understanding. Our moto is - You just play, we do the rest.
In this talk, I’ll walk through the architecture and design of the current version of Tabbers, a project that uses modern (and less modern) AI techniques to process both video and audio of guitar performances.
The system includes:
• Audio analysis to detect played notes and chords
• Computer vision to detect finger placement and fretboard position
• Fusion pipeline for aligning visual hand positioning with note transcription
• Tablature generation engine
• Web app of a player for practicing with variable speed and looping segments (and unfortunately, for editing detection mistakes)
We’ll dive into key technical challenges like handling polyphonic audio, fretboard tilt, aligning asynchronous audio and video, and distinguishing techniques like slides or bends - and how we solved them using AI-first design thinking.
Tools & Stack:
• Backend in Python (FastAPI, OpenCV, Librosa, Pandas, Numpy, etc.)
• Basic-Pitch by Spotify, Ultralytics YOLO Models for CV, MediaPipe for hand tracking
• Pyppeteer, ffmpeg
• GCS for all cloud needs
• Next.js frontend with custom SVG rendering for interactive tab playback
Who Is This For:
• PyData developers interested in applied ML for video/audio
• Musicians exploring generative tools
• Engineers looking to build educational interfaces powered by AI
What Attendees Will Learn:
• Basics of Audio and Video analysis
• Trade-offs between heuristic methods and ML in "free-tier-solopreneur" mode
• How to turn a hobbyist tool into a structured AI product
No previous knowledge expected
Lior Kupfer, creator of Tabbers. With a background in AI, Applied DS, Analytics, and Product Development and Management, I’m passionate about building smarter, user-friendly systems that create real impact.