NVIDIA NVIDIA
May 13, 2026

Snap’s GPU-Accelerated Secret to Processing 10 Petabytes a Day | NVIDIA AI Podcast Ep. 298

"Snap processes more than 10 petabytes of experimentation data every single morning—and with NVIDIA GPU-accelerated Apache Spark on Google Cloud, Snap cut job costs by 76%, reduced memory usage by...

"Snap processes more than 10 petabytes of experimentation data every single morning—and with NVIDIA GPU-accelerated Apache Spark on Google Cloud, Snap cut job costs by 76%, reduced memory usage by 80%, and eliminated 120 terabytes of disk spill from its pipelines.

Prudhvi Vatala, head of engineering platforms at Snap, joins the NVIDIA AI Podcast to break down how he and his team completely modernized data infrastructure for a social platform serving nearly a billion monthly active users—using NVIDIA cuDF plugin (formerly referred to as NVIDIA RAPIDS plugin) for Apache Spark on Google Kubernetes Engine, with zero application code changes.

🔬Topics covered

How Snap runs A/B tests at planetary scale using rigorous statistical methods like heterogeneous treatment effect detection and variance reduction

Why Snap reuses idle inference GPUs between 1–5 a.m. for batch data processing—and how it built a Kubernetes-based platform to do it

How NVIDIA cuDF delivered 3x+ speedups on join-heavy Spark jobs with no code rewrites

The full business impact: 76% cost reduction, 62% fewer cores, 80% less memory, 120 TB of spill eliminated

How a three-way partnership between Snap, NVIDIA, and Google Cloud made it possible in just 8–9 months

Chapters
0:00 Introduction and Snap overview
3:35 What is Snap’s experimentation platform?
4:05 Why experimentation, safety, and privacy are core at Snap
4:52 How A/B testing works at billion-user scale
8:14 Discovering NVIDIA cuDF plugin
9:06 Benchmarking results: join, union, and aggregation jobs
12:00 Reusing idle GPUs overnight via GKE
13:24 Building a bottom-up GPU data platform at Snap
17:48 Results: 76% cost reduction and partnership impact
20:56 Snap’s evolution and what’s next

Learn more
NVIDIA cuDF
102

Comments 0

Replying to
5000 max

Be the first to comment!

This website uses cookies
This website uses cookies.
To maintain our services, we use cookies. They allow us to personalize ads, analyze our traffic, and provide social media features.

Customize your preferences

Select the cookies you accept: