Skip to content

ODSP2172 Pre-recorded On demand

AI-scale growth: How Hugging Face and Instacart scale in-memory data

Oded Poncz

Summary

From Microsoft's AI summary

Oded Poncz of Dragonfly argues that AI agents make users effectively multi-threaded, so infrastructure demand may rise 10x to 100x in concurrency and throughput, with real-time data storage up tenfold. He cites GitHub, which planned for 10x capacity in October 2025 and within five months concluded it needed 30x. Instacart reportedly cut hardware use by 70% for its feature store, moved to one cluster and more than doubled capacity on Dragonfly. Feature stores have grown from about 100 GB to tens of terabytes, at roughly 30% annually. No availability or pricing was stated.

Session description

Microsoft's catalogue description

AI companies can experience rapid growth that places new demands on in-memory data infrastructure, from thread bottlenecks and resharding to tail latency and backup memory use. Explore how two teams address these challenges. Hugging Face split its Redis setup as the Hub reached 3 million models, then adopted Dragonfly to simplify operations. Instacart replaced 348 ElastiCache nodes with 100 Dragonfly nodes and reduced average and P99 latency by 40–50%, with no customer impact.

← All sessions