PDF 原檔：報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_original.pdf
講者公開簡報（OCP 官方 Recorded Content 頁提供下載）。文字以 PyMuPDF 逐頁抽取，
### p<n>為頁碼；圖表內容見原檔。
圖片清單（已驗證 2026-10-19）
| 檔名 | 頁 | 分類 | 親眼所見內容 |
|---|---|---|---|
報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_p4.png |
p4 | 真資料圖 | TPS per MW（縱軸）對 TPS per user（橫軸）概念曲線，分三區：Search/Chatbots、Deep Research and Planning、Agentic Coding and Automation |
報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_p6.png |
p6 | 真資料圖 | NeMo Switchyard 路由架構與成本表：Agent 經路由分配至 Opus 4.8 30%、Nemotron Nano 22%、Gemma 4 26B 13%、Qwen 3.6 35B 35%；單一模型完成 89 任務花 $179，Switchyard 完成 90 任務花 $60 |
報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_p10.png |
p10 | 真資料圖 | NVL72 機櫃、NVLink spine、NVLink 6 Switch Tray 實物照；標示 3x 低延遲、10x 封包率、130 TFLOPS 網內運算 |
報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_p11.png |
p11 | 真資料圖 | Agentic 負載變化：ISL 中位數 1K→142K、快取 token 60%→98%、輪次 3→65、子代理 0→4；來源標 SemiAnalysis AgentX |
報告_OCP2026_NVIDIA_AI_Tokenomics_20261015_p13.png |
p13 | 真資料圖 | GB300 NVL72 對 Vera Rubin NVL72：每 MW token 吞吐曲線（標 2x／10x／30x）與每百萬 token 成本曲線（標 45x lower），橫軸 TPS/User 50～300 |
原始內容
p1
Shruti Koparkar Sr. Manager, Product Marketing, NVIDIA Eduardo Alvarez Manager, Technical Marketing Engineering, NVIDIA AI Tokenomics Explained AI CLUSTERS
p2
Two Key Levers Impacting Economics of AI What tokens you choose? • Model complexity • Context length • Interactivity What infrastructure you choose? • Infrastructure cost • Throughput per Watt • Token Cost
p3
What Tokens You Choose: Not All Tokens Are Created Equal Model Size and Complexity Tokens produced by more complex models carry more intelligence Long Context Tokens produced with longer context carry more intelligence High interactivity Tokens that arrive faster are more valuable
p4
Map Usecases To Optimal Token Value 0.0 M 0.2 M 0.4 M 0.6 M 0.8 M 1.0 M 1.2 M 1.4 M 1.6 M 1.8 M 100 200 300 400 500 600 700 800 900 1,000 TPS/User Throughput (TPS per MW) More responsive user experience Slower, high Volume, basic models/post-trained SLMs Fast, Lower Volume, frontier models Near Instant, Very Low Volume, smartest models Search, Chatbots Deep Research and Planning Agentic Coding and Automation More responsive and intelligent user experience Interactivity (TPS per user) + Larger Models and Context
p5
What It Looks Like In Practice? Right model for right task Use premium models when necessary Real-time routing Intelligent routing based on model fit, pricing, and context Governance and observability Build a control plane that provides visibility into token use
p6
NVIDIA NeMo SwitchYard: 3X Lower Cost Per Agentic Task Agents Agents Agents Agents Agent Nemotron Nano Opus 4.8 Qwen 3.6 35B Gemma 4 26B NeMo Switchyard 30% tasks 22% tasks 13% tasks 35% tasks Single model NeMo Switchyard LLMs 1 4 Tasks completed 89 90 Total cost $179 $60 Configurable family of routing algorithms Works with open and closed models Compatible with leading model APIs, agents, SDKs and frameworks Open source model routing library for AI agents helps map agent needs to model capabilities
p7
What Infrastructure You Choose: Maximize Tokens, Minimize Cost Lowest Token Cost Maximum Token Output Model and Workload Efficiency System Efficiency Software Efficiency
p8
Model Efficiency: Mixture of Experts Router Expert 5 Expert N Expert 1 Expert 2 Expert 3 Expert 4 Output + Feed Forward Network Norm + Attention Norm Input Kimi K3 2.8 Trillion parameters, 108 Billion active 896 experts / 16 activated per token Lower per-token compute and HBM bandwidth But…
p9
Communication is Hidden Cost of Mixture of Experts Router 4 Router 5 Router N Router 3 Router 2 Router 1 Expert 5 Expert N Expert 1 Expert 2 Expert 3 Expert 4 But… Heavy all-to-all traffic for multiple tokens across multiple batches Strong dependence on scale-up network bandwidth Output + Feed Forward Network Norm + Attention Norm Input
p10
System Efficiency: Fast Inter-GPU Communication with Purpose-built Scale-up Networking 3x Lower Latency 10x Higher Packet Rate 130 TFLOPS In-Network Compute NVLink 6 Switch Tray NVLink Spine NVL72 Rack 72 GPU scale-up domain
p11
Agentic AI Is Significantly More Demanding Number of Turns Session-aware prefix-cache retention Interval-based and selective checkpointing 3 → 65 Cached Tokens Prefix caching Distributed KV caching KV-cache offload KV-aware routing 60% → 98% ISL (median) Kernels for broader set of shapes/sizes Disaggregated serving with rate matching 1K → 142K Sub-Agents Intelligent scheduling and routing KV cache needs to be preserved for longer 0 → 4 Prompt Orchestration CPU Agentic Tools and Skills CPU CUDA LLM Memory—Context Window Security and Governance CPU Context Observe Reason Act LLM LLM LLM LLM DPU CUDA LLM EW NETWORK Source: SemiAnalysis AgentX workload characteristics
p12
Software Efficiency: Enable Every Optimization Serving and Orchestration Application Acceleration Infrastructure Access Distributed Serving Dynamo llm-d Memory Management KVBM Orchestration Grove/K8s Runtime SGLang TensorRT-LLM vLLM KV Cache Transfer Mooncake Collective Communication NCCL NIXL Storage DOCA Memos DPU DOCA GPU CUDA Run.ai LMCache Accelerated Kernels CUTLASS FlashInfer Model Compression ModelOpt CUDA Libraries cuDNN cuDF cuBLAS Disaggregated Serving KV Cach Offloading Topology Aware Scaling Kernel Fusion Compute-Comms. Overlap Fast Model Loading Low Precision Kernels Inference Optimizations KV-aware Routing NIXL Speculative Decode
p13
Extreme Co-Design Delivers Performance Gains 0M 20M 40M 60M 80M 50 100 150 200 250 300 Interactivity (TPS/User) Disaggregated Serving Rate Matching Large-scale Expert Parallel Distributed KV-caching KV-cache Offloading KV-aware Routing Fused CUDA Kernels $0.00 $0.10 $0.20 $0.30 $0.40 50 100 150 200 250 300 Interactivity (TPS/User) GB300 NVL72 30x Vera Rubin NVL72 45x Lower GB300 NVL72 Vera Rubin NVL72 Total Token Throughput Per MW Cost Per Million Tokens
p14
Optimize Model And Infrastructure For Your Usecase Lower tokens per task Lower cost per token Lower cost per task Mix of open and proprietary models Performant infrastructure right-sized to your use case
p15
Thank You!