In Episode 4 of AI Factory Insider, host Kaushik Shirhatti (VP, AI Factory, NVIDIA) is joined by Shruti Koparkar (Sr. Manager, Product Marketing, NVIDIA) and Tue Tran (GTM Strategic Ops, NVIDIA) to discuss what a token actually is, why not all tokens are equal, and how to tie AI factory investment to real business ROI.
What we cover
0:00 — Introduction
1:28 — Meet our guests: Shruti Koparkar & Tue Tran
2:42 — What is a token?
3:54 — What is "tokenomics," and why is it relevant now?
4:58 — Not all tokens are equal: intelligence, context, and speed
7:44 — Trading vs. research: matching tokens to the use case
9:20 — Retail video: one-token answers vs. batch captioning
10:40 — Sizing token demand: back-of-napkin math
12:10 — The multipliers: reasoning, agentic loops, and cache hit rate
13:54 — KV cache and prefix matching explained
16:34 — Supply: the AI factory as a car engine
17:29 — Dense models vs. mixture of experts
20:19 — Right model for the right task: a telco example
21:24 — Intelligent routing with NeMo Switchyard
22:10 — Where are your tokens going? Observability with NeMo Relay
22:38 — Matching infrastructure: NVL72, HGX, and RTX PRO Servers
23:31 — Dynamo and extreme co-design
24:25 — Monetization: four paths from tokens to ROI
27:54 — Just get started, and make it a team effort
29:18 — Guardrails for a token tsunami
31:32 — Wrap-up
Resources
AI Tokenomics: A Framework for Deploying and Monetizing Inference at Scale
Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value
NVIDIA AI Factory Validated Design Guide
What we cover
0:00 — Introduction
1:28 — Meet our guests: Shruti Koparkar & Tue Tran
2:42 — What is a token?
3:54 — What is "tokenomics," and why is it relevant now?
4:58 — Not all tokens are equal: intelligence, context, and speed
7:44 — Trading vs. research: matching tokens to the use case
9:20 — Retail video: one-token answers vs. batch captioning
10:40 — Sizing token demand: back-of-napkin math
12:10 — The multipliers: reasoning, agentic loops, and cache hit rate
13:54 — KV cache and prefix matching explained
16:34 — Supply: the AI factory as a car engine
17:29 — Dense models vs. mixture of experts
20:19 — Right model for the right task: a telco example
21:24 — Intelligent routing with NeMo Switchyard
22:10 — Where are your tokens going? Observability with NeMo Relay
22:38 — Matching infrastructure: NVL72, HGX, and RTX PRO Servers
23:31 — Dynamo and extreme co-design
24:25 — Monetization: four paths from tokens to ROI
27:54 — Just get started, and make it a team effort
29:18 — Guardrails for a token tsunami
31:32 — Wrap-up
Resources
AI Tokenomics: A Framework for Deploying and Monetizing Inference at Scale
Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value
NVIDIA AI Factory Validated Design Guide
This website uses cookies
Comments 0
Be the first to comment!