Eventual Blog
What I learned re-evaluating VLADBench on the latest VLMs
GPT-6 Astra and Claude Opus 5.5 top the leaderboard, but the cost-performance frontier is a more interesting story.
Cutting dead frames from DROID: 500 hours of robot data in 32 seconds
The robot state recorded next to the video tells you which frames matter before you decode any pixels. Trimming all of DROID takes 32 seconds on a laptop.
How we made our LeRobot video reader up to 15× faster
LeRobot has emerged as the dominant open format for robot learning data, but decoding frames is expensive. Here's how we made Daft's native LeRobot reader up to 15× faster.
Finding a Needle in the Haystack: Querying Physical AI Data with Daft
Pose + semantic search over Apple's EgoDex hand-manipulation dataset with Daft: SigLIP embeddings meet hand-pose geometry. Ctrl+F for physical AI data.
Disk is the data plane: Flight Shuffle in Daft
How Daft rebuilt distributed shuffle around Arrow Flight, local disk, and streaming reads to handle multi-terabyte workloads.
Scaling As-of Joins
How we built, broke, and re-built our ASOF joins — 5.5x faster, half the memory of pandas, and scaled to a distributed cluster.
Image Embeddings: Tutorial & Examples
Learn about the concept of image embeddings, their various use cases, and best practices for handling them in data processing workflows.
Multimodal Embeddings: Tutorial & Examples
Learn multimodal embedding techniques for cross-modal search, recommendation systems, and content moderation applications.
Tuning Daft's Distributed UDFs: Lessons from ByteDance
Learn from the ByteDance Volcengine LAS Team on how to optimize Daft UDFs on Ray. Discover the formula to evenly distribute data across actors.
Introducing Dynamic Batching: Auto-Tuning for Daft Pipelines
Manually tuning batch sizes is hard. So I implemented dynamic batching to never deal with it ever again.
How We Use AI Coding Agents
Our engineering team's best practices for working with AI coding agents.
Multimodal Structured Outputs: Evaluating VLM Image Understanding at Scale
Leveraging ablation for contrastive image understanding evaluation in Daft
Processing 99% of U.S. Caselaw for Under $1 in the Common Pile
How Teraflop AI processed 7 million court documents and 40 million pages spanning 365 years of U.S. caselaw for under a dollar using Daft.
Prompting with DataFrames: Massively Parallel LLM Generation is Here
Discover how Daft's prompt function revolutionizes LLM workflows with massively parallel context engineering on DataFrames.
Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale
Learn how Dynamic Prefix Bucketing reduces LLM batch inference time, improves throughput, and unlocks faster multimodal processing at scale.
Simplifying Voice AI Analytics with Daft: Transcription, Summaries, and Embeddings at Scale
Build a Voice AI analytics pipeline with Daft and Faster-Whisper to convert raw audio into searchable transcripts, summaries, and embeddings at scale.
Using PyTorch DataLoaders to Streamline Multimodal Data
Learn how PyTorch's DataLoader streamlines deep learning pipelines by efficiently loading and shuffling data in batches.
Benchmarks for Multimodal AI: Spark, Ray Data, and Daft
Multimodal AI workloads break traditional data engines. Daft ran 2-7x faster than Ray Data and 4-18x faster than Spark while finishing jobs reliably across audio, video, document, and image workloads.
Introducing Flotilla: Simplifying Multimodal Data Processing at Scale
Flotilla, Daft's new distributed engine, processes terabytes of multimodal data in a single query up to 18x faster than Spark and Ray Data, while running efficiently, reliably, and without manual tuning.
Exploring Daft's Local Execution: The Swordfish Engine
Explore how Daft's Rust-powered engine executes DataFrame and SQL queries. Learn how Swordfish enables fast, streaming image processing at scale.
After the First Run
Using Daft's observability tools to uncover performance pitfalls
Making GPUs Zoom (Part 1)
How Daft is approaching large-scale model inference with advanced GPU optimizations for faster multimodal AI workloads
End-to-End Distributed PDF Processing Pipeline
Build production-ready PDF processing pipelines with distributed computing, OCR, spatial analysis, and GPU embeddings
Embedding Millions of Text Documents With Qwen3
Learn how to achieve near-100% GPU utilization processing millions of text documents with Qwen3 embeddings.
Processing 300K Images Without OOM
A Streaming Solution
We cloned over 15,000 repos to find the best developers
An adventure in AI and data engineering to analyze developers across Github
High-Performance File System Support With DeepSeek 3FS
Learn how Daft integrates with DeepSeek SmallPond 3FS to deliver faster file access and efficient data handling for modern workloads.
Adversarial file reading: from 10,000 small CSVs to massive Parquet files
Learn how adversarial file reading speeds up data ingestion at scale, enabling fast conversion from thousands of CSVs into efficient Parquet files.
Working with the Apache Parquet file format
This guide shows how Apache Parquet boosts read performance, lowers storage use, and supports efficient workflows for large analytical datasets.