Cutting dead frames from DROID: 500 hours of robot data in 32 seconds
The robot state recorded next to the video tells you which frames matter before you decode any pixels. Trimming all of DROID takes 32 seconds on a laptop.
Turning robot video into training-ready data with daft-physical-ai
A Python library built on Daft for turning robot video into training-ready data, starting with hand tracking and reward scoring as UDFs, with more to come.
How we made our LeRobot video reader up to 15× faster
LeRobot has emerged as the dominant open format for robot learning data, but decoding frames is expensive. Here's how we made Daft's native LeRobot reader up to 15× faster.
Finding a Needle in the Haystack: Querying Physical AI Data with Daft
Pose + semantic search over Apple's EgoDex hand-manipulation dataset with Daft: SigLIP embeddings meet hand-pose geometry. Ctrl+F for physical AI data.
Disk is the data plane: Flight Shuffle in Daft
How Daft rebuilt distributed shuffle around Arrow Flight, local disk, and streaming reads to handle multi-terabyte workloads.
First-class observability in Daft
A new dashboard, per-operator memory attribution, and OTel endpoints for your existing collector. Everything you need to see what Daft is doing with your query.
Scaling As-of Joins
How we built, broke, and re-built our ASOF joins — 5.5x faster, half the memory of pandas, and scaled to a distributed cluster.
daft.VideoFile: Seek Lazily, Get Frames
daft.VideoFile decodes only the frames you need. Keyframes, time-sampled, or windowed seek, built for robotics datasets, dashcams, and moderation queues.
Daft Extensions Featuring daft-h3: Native Rust Performance, Community Owned
Daft now supports native extensions via Apache Arrow's C Data Interface. daft-h3 is the first community extension — 9 Rust-native H3 geospatial functions, 3–16x faster than Python UDFs.
Audio transcription at scale with daft.AudioFile
How to transcribe thousands of audio files with Whisper using daft.AudioFile — handling resampling, silence splitting, and worker-resident model loading without the boilerplate.
Image Embeddings: Tutorial & Examples
Learn about the concept of image embeddings, their various use cases, and best practices for handling them in data processing workflows.
daft.File: Lazy Metadata Filters
Filter millions of files by path, size, and content type before opening any of them. Cheap operations first, expensive operations on the survivors.
Multimodal Embeddings: Tutorial & Examples
Learn multimodal embedding techniques for cross-modal search, recommendation systems, and content moderation applications.
Daft UDF Patterns: Four Patterns, One Notebook
Row-wise, generator, async, and stateful UDFs — one notebook, one dataset, runnable side by side.
GPU Inference with @daft.cls
Run GPU models on millions of rows without OOM. Real patterns from ByteDance, Essential AI, and more.
Stateful UDFs with daft.cls: Python Classes that Scale
Turn any Python class into a distributed operator. Hold models, connections, and clients across rows with one decorator.
Stateless UDFs with daft.func - four patterns, one decorator
Row-wise, async, generator, and batch UDFs in Daft — one decorator, zero boilerplate, local or distributed.
Daft UDFs: What is a UDF and why do you need one?
Daft User Defined Functions (UDFs) let you run custom Python inside a distributed DataFrame pipeline. Leverage Row-wise, Async, Generators, and Batch.
Newsletter
Get new posts in your inbox
Engineering, product, and company updates from Eventual.