video_OCP2026_NVIDIA_主題演講_AI工廠規模化_20261015

Thank you. Well, I wrap up today, so I won't keep you too long from dinner, but of course, this is OCP. So, I'm here to have a chance to really geek out on the hardware and all the things that we've been doing at NVIDIA and contributing back to OCP. Uh Our history of of contributions dates all the way back to 2013, the original uh Mellanox days into today the GPUs when they were still PCIe cards to our first GPU motherboards with HGX all the way up to Blackwell, what I talked about last year. This year, uh I'm going to talk a little bit about Rubin, but the span of what we built all the way back to those contributions from those first GPUs in 2015, the networking in 2013 to today has built an entire ecosystem and infrastructure of open computing that the world is benefiting on. And it's infrastructure and in that it's infrastructure. It is existing, it's lasting like bridges, like cables, like pipes. This is computing that is available to the world even before AI and AI today. We now have over 10 million developers developing on accelerated computing infrastructure. And every CSP, NCP, every OEM, every ODM, they're of course you're all here. There's over 300 major companies in the supply chain alone building products, components, pieces from OCP in building this ecosystem. I certainly have to thank all of you for that rich ecosystem and the support for all those developers. That makes that infrastructure and lasting fungible. It can do pre-training, it can do post-training, it can do inference, it can do agentic workloads, and of course, it can do accelerated computing. These that infrastructure, that that every model, every workload, whether it be closed or frontier, is available built on OCP for well over a decade worth of open contributions. And with every release, we make it faster. I talked to one customer recently. They they bought some Voltas back in 2019. They they rented them out for about a quarter an hour. Today, they're now renting them out for about a dollar an hour. That's infrastructure. That makes it gets faster, affordable, it lasts for uh literally almost forever now, and makes um help build money and help builds the growth of what we're all doing together. The latest generation, of course, Vera Rubin it is with every generation, we sit down and we're trying to figure out not just how to make something faster, more capable uh with more performance for the the agent or the user or the AI model or the simulation, but also more affordable. Vera Rubin, by going end-to-end, working on disaggregated serving, studying rate matching, all the software, all integrating all the latest KVE cache implementations, and building up that MGX rack, which now is OCP, you get 2x, 10x, at the extreme 30x more token performance experienced by the user. But on the right-hand side, we're actually redu- at same time reducing the cost of those tokens. Because there's that much more computing available in the rack, we can serve more and actually reduce the cost of those tokens by 45x. This is measured on actual benchmarks, in this case the Agent Agent X benchmark done by Semi Analysis. So, we're here at OCP. Vera Rubin is in full production, as you've probably seen. It's it's showing up in all of the world's data centers, at all the world's CSPs, and we're already seeing some amazing results from it from our customers, being deployed everywhere. Uh and NVIDIA, just like we did with Blackwell last year, we are contributing our third generation MGX rack architecture, which is what with Rubin's based on 2OCP. This is our MGX 1.2 factory scale high-density liquid-cooled architecture. And everything from the design of the compute tray itself to the liquid-cooled bus bar the way to the in-tray MQDs and MQDBs quick disconnects. I have learned enormous things about water flow and how to make things run run smoothly. Um the blind mate manifold, the rear rack extenders, the the whole base system, and also standardizing helping the world standardize on flow rates so we can all be building components and pieces and parts that are compatible not just with NVIDIA's product but every member of OCP can get the benefit and we can all be designing for today's data centers and for the next generation. This is our commitment to OCP and and commitment to all of you to continue to make MGX OCP compatible and contribute it every year with every generation. And this year not just all those components, but we also focused on manufacturability. On the left is the compute tray, the GB200 and BL72. And you can see going to Vera Rubin the changes. We eliminated every cable. It actually takes took 90 minutes to make that compute tray with Grace Blackwell. We learned a lot. We optimized. A lot of it was liquid cooling and QD connectors and how to make everything reliably and stable and fully automated. Factories today both here in the US and of course in Taiwan and Asia and around the world are now manufacturing the Vera Rubin tray. It takes 1 minute to assemble and manufacture that compute tray. And we're literally producing thousands of racks a week at this point. This contribution of course is great for OCP. All of we're sharing all of our learnings with the rest of you so that we can continue on this journey and scale up. In addition, we're planning for the future. This is the first 800-V DC AI factory that's being designed and built now, engineered for the next decade of accelerated computing. It's actually being built by Vision Bay.ai, and it is actually taking the OCP 800-V DC standard that's being developed now and implementing the the first version of it in this data center. The benefit of 800-V of course is we're doing fewer transitions and stepping. That means less components that could potentially fail, and we're increasing the efficiency of that data center. Not to mention, we're using copper, a lot less copper. Uh 45% uh less copper is required with 800-V than than alternatives. And it's also scalable. This can support the 100-kW racks that we have today, um but can actually scale all the way up to 1-MW volt racks by delivering 800-V power direct to to the to the data center and across all those rows. We're super excited about this. We've actually uh contributing this along with our partners. We have a blog on our our website, uh and to prove out and to validate an 800-V DC infrastructure system that's going to help scale data centers today with Rubin and Ultra out to Fineman and beyond for for both our products and both for the entire ecosystem. Okay, well, everyone's talking about CPUs. Of course, Nvidia also has um been building CPUs for quite some time. We built Grace and paired it with our Blackwell and Hopper systems, if you remember. And with Rubin, we built a new CPU, the Vera CPU. Uh we've used it with our Vera Rubin GPUs. But we also, like everyone else here, see the opportunity for building a new kind of CPU for a gentle computing. In the pre-cloud space, this is in PCs and in gaming, we had fast CPUs. They were clocked at gigahertz of performance. Maybe only had a few cores, uh eight or 10 or 12 or 16, uh but they were fast, and they got excellent single-threaded performance for doing client-based workloads at the um but they weren't uh at the cloud scale. Come the cloud where everyone's buying dollar per core and optimizing for cost, the opportunity was to build uh lots of throughput, lots of cores, but not necessarily have the highest of clocks. Vera is actually a different kind of CPU. It was designed to take We care about single-threaded performance. If you're running a GPU, running an AI factory, you for gigawatts, costing billions of dollars, you do not want to wait on any CPU execution. You want to keep those GPUs busy. But you also need throughput. We have all those tasks we have we have up to we have lots of different operations that have to happen to keep those GPUs busy. So we need both fast single-threaded performance, but with all the cores running all the time. And that's actually what Vera uh was designed to do. It was designed, of course, to be a great partner for our GPUs, but it's also a great partner for doing all the agent tool calling. It actually is double the uh si- per core performance compared to comparative Grace. It's an NVIDIA core design. Uh it is 3.4 terabytes a second of core-to-core bandwidth. We We may not have as many cores as the the cloud dollar per core CPUs, but what we did spend it on is the communication of all those cores to each other and to the memory. There's no compromises, no chip attacks, fully square coherent fabric. So if every core wants to do database operations, it's not going to hit a memory wall, it's not going to hit a transition, and it can fully access all the resources it needs. As a result, we also combine it with LPDDR memory. This is memory that was designed for cell phones that we've hardened for the data center. We innovated this with our Grace CPUs, uh and we've been using it in an embedded space for quite some time. Adding ECC, working with the memory vendors, working on new standards for uh low energy, high bandwidth, low latency uh LPDDR memory. This CPU actually has 1.2 terabytes a second per, uh, per core and it get all of its work done that comes through that fabric. You see that in the results and benchmarks. Uh, we ran Meta's, uh, DC perf benchmark, which is an open source benchmark. It's on GitHub. It, uh, was representative of their data center nodes. Saw 2x more uh, better performance with Vera compared to compared to Grace and actually Meta just recently announced that they'll be building their first Vera cluster uh, like they did with Grace uh, early next year. Uh, on simulation and HPC and running agentic uh, codes for for uh, industry use cases, 2x on the HPC golden suite that's been measured in in supercomputing and HPC community for some time. And even for uh, operations like Apache Kafka, databases with Red Panda, 3.5x. You can read about all this uh, but it's shipping now. We have actually been delivering them to our customers and actually building them out in inside the clouds. And what we're also announcing here at OCP is that the the new Nvidia Vera CPU MGX rack is now available and shipping. Uh, this is OCP compatible design. Uh, our partner, uh, the same leverages all of the same ecosystems that we talked about with Vera uh, Rubin as Vera CPU. This rack actually has 256 CPUs in it, all liquid cooled, all 1U, highly dense configuration. Uh, it supports all of the agentic tool calling and all of the AI safety that is that can be implemented with our BlueField uh, front end NICs to secure those agents and make sure they stay within their sandbox and don't do anything that operators don't want them to want them to do. Uh, the early adopters you can see here, but the ecosystem partners what I'm really excited about with everybody here building this rack and it's now shipping. Again, it's cableless, it's modular, it's liquid cooled and it's because it's OCP, it's open. It's fully customizable. You can take and design it with a GPU tray in there, a CPU only, we can do a 2U, vary the density including offer both, you know, standard PCIE accelerators with various CPUs and mix and match, or just go with standard 2U air cooled. All of this is available and shipping now. Finally, OCP is not just about the brass iron and the hardware, but we all are building data centers. And you heard some great talks from Santosh and others here that are building some of the most amazing uh data centers and training some of these amazing models. It's a great privilege to work with all of them uh to drive performance, MTBI, and in the end uh token efficiency to generate the AI and the tokens that we're all using, our agents are using, to bring forward this revolution. We have a a platform called DSX. It is our software and ecosystem and a reference architecture for a modern um accelerated AI factories. We have actually now contributed is our reference implementation, how we recommend how you build a full mod at scale complete AI factory with NVIDIA technology and made it open and contributing it to OCP. This is actually uh to build out to be able to do the state of the art 350 kW per square foot, 100% liquid cooling, uh all of the techniques and designs and and scaling and specifications for flow and other things in collaboration with AMD, with Google, with Meta, and of course with NVIDIA to build an open data center spec. We're we're joining that and contributing our reference architecture to OCP. We're also working on how these data centers want to talk to the grid. You know, AI factories need to be part of the grid, not just consumers, but contributors. And how do we want to manage these quite powerful assets and capabilities and factories but not affect ratepayers and and and be a burden on the grid but an asset. So, we've got to manage and distribute and and take responsibility. If the grid needs more power, these factories need to respond and be able to dial in the power, maybe dial it down or change their workloads or push workloads to tomorrow instead of today because it's hot outside. Uh this is actually a collaboration with Google, again Meta, Microsoft, and Nvidia to design a grid resiliency specification that actually works. And that's a big part of what we're doing here at OCP to be open. Uh utilities are the most open thing we have out there. Has to work with open standards so that we can all be part of the grid and not not a burden. And finally, the software, the actual operating system in the data center. Uh DSH has lots of different components for managing and deploying uh uh parts of the fleet. One of the ones I'm most proud of of is actually called Max LPS. Max LPS basically says, "Let's be smart smart about the power we use." We typically hear about a 100 MW data center, but we don't actually use all 100 MW all the time. We may be provisioned that by the utility, but the actual usage every workload, whether it be running um GPT or doing a database operation, is going to consume a different amount of power for the rack. We're not running 100% all the time. That's an opportunity. The world is power constrained right now. We can't get uh data centers being stood up, but if we go and study and look at the data centers we already have, we're actually not using all the power that we have provisioned to us. Max LPS is a way to actually take advantage of that opportunity. It's software that's integrated into our rack and power management system and open source software that actually helps coordinate and pull all the different workloads and integrating with the schedulers to say, "Hey, there's actually strain of power here. You can put some more workload on this GPU." Or even better, you can actually potentially put more GPUs in your data center because you don't need to worry about the max performance and and that TDP of the actual rack, but actually what's the working power of that rack? And as long as the software there is is constantly power capping and managing so that no circuit blows, and we can stay within spec, we can actually increase the number of GPUs and number of tokens we're building. This is a project being done with Nscale right now with a data center in I believe it's Iceland. They've actually been able to put 37 more GPUs in the data center by deploying the Max LPS software, again open source software, into that same data center. And they they target toward the most highest revenue opportunities they had inside of their their data center to maximize the revenue and actually grew their revenue and their their token throughput by 50%. I think all of us need to look at what is your not the ability to deliver compute at a maximum power, but how can you manage your power efficiently? How is your management controller working? Can you respond quickly? And are we designing for the flexibility of these workloads? There isn't just one AI model that's going to rule the world. There isn't just one tool call that's going to rule the world. It's going to be dynamic. Our systems, if we can take advantage of this, we can actually more efficiently use the data centers and the powers that we have now and quite literally increase the number of GPUs by another 40% today. Check out Max LPS, understand how your software, how your control logic can manage and and operate a dynamic power environment and to maximize your token throughput, your revenue just like Nscale did in this data center. Okay, networking. We have obviously been pioneering in the world's best networking and ethernet for AI. Our latest networking that comes with bare ribbon is the new Spectrum X photonic multi-plane ethernet. This actually is a our first CPU based ethernet switch. Got Spectrum built on Spectrum X. It works in sync with the ConnectX-9 SuperNics, which I see there, and the BlueField-4 DPUs to manage and connect, as we just heard, the scale out and of course the scale up of all and across and everywhere for running agents, training models, and doing inference. We this the what's interesting and exciting about the the new latest Spectrum X is actually it's multi-plane support. Multi-plane allows it has hardware managed capabilities to detect fault detection, so it can find a link down or a link flap actually within 2.6 milliseconds. Prior network switches in the rest of the world has to use software to detect it and respond to link downs. If you have 2.6 milliseconds, you can respond in a in a in like a hundredth of a second, very quickly identify a link flap, and take that link out of the chain so that you're not losing data or slowing down the entire competition. You will respond immediately. Also using multi-plane and hardware system multi-plane with the with the fault detection, you can actually use fewer 40% fewer switches. What's even cooler is that this is a co-packaged optical switch. That means we put the transceivers and the optical engines uh directly on the package, co-package it with the switch. You can see it there. That actually delivers you delivers you 5x higher power efficiency, and because we've packed co-packaged it and designed it, the signals only need to travel from the switch chip right to the optical engine, which is on the same package. In the middle of that switch actually is is NVIDIA's CPU silicon. There's a shot of it here. Um this is the CPO Spectrum-X uh package that actually is part of the Spectrum-X Photonic multi-plane switch. Uh in in this configuration, we have a single switch. In the larger Spectrum-X, we actually have four of these chips to build up a 400 terabit uh spectrum uh switch solution. What's super exciting about this? This is actually shipping this year. So, super excited to be announcing this. It's going to totally change the reliability and the scale of which we can do training at scale and actually connect the world's AI data centers together. Very excited about it. Thank you. >> [applause] >> Finally, we of course make this open. Spectrum is open to OCP and open to integrations with all of our CS uh partners. Happy to be announcing our work with Meta and Santosh and his team to actually deliver the uh Spectrum-6 Ethernet switch for Meta's AI factory. This is the um the Meta 1.2T F uh switch called the IceCube. Uh it's based on NVIDIA Spectrum-X, actually implements um the Meta's own uh networking protocol to connect these at scale, and actually implements the Spectrum-X multi-plane uh in collaboration with them for better resiliency and smooth deployment. In addition, we've been working with Google. Last year, Google contributed Google Falcon, the transport engine, to OCP. A really important contribution that lets the uh share Google share how they actually connect all of their data centers and how they do the transport across all those switches. Google Falcon. We've been working with them and partnering with them actually to port and implement uh Google Falcon on NVIDIA. So, we're now we're also contributing and announcing that we're actually we've done that. We put all the multi-path load balancing, the programmable congestion control, all the hardware loss recovery, and the per connection security that Google Falcon has, and made it available on the Spectrum X Supernic, and implemented on the BlueField DPUs, and contributed that as well back to OCP. Spectrum X is our platform for open networking. Whether be Google Cloud Falcon, Meta Rocky, uh Open AI's MRC, or Nvidia's own Spectrum X adopted RDMA. Uh there's lots of amazing ideas and ways to actually scale and connect all of these data centers, whether they'll be running AI or accelerated computing. And to design and it's a platform for innovation. And it's and it's all open, which is what's what's so great about OCP and what we're here And finally, one more thing. We're not while the world is actually now receiving all of their their Ribbon centers and building out, and you've already seen some of the amazing work of you know, Astra is now running on Vera Ribbon, it's and running faster. We're not sitting still. We're already working on the next generation. Here's a sneak peek of Vera Ribbon Ultra. This actually is not 72 GPUs in a single rack, but actually 576 GPUs that are all NVLink connector. This is using our next generation NVLink CPO switch tray, and I have a picture of it here, uh along with our next generation Vera Ribbon tray. You may notice some similarities here. This is all based on the OCP standards. And it's actually going to be connecting all of those GPUs and all of those individual racks together with CPO networking to build one giant 576 GPUs all connected with NVLink, with the little little yellow fibers there connecting those optical engines. This will be the highest performance per watt um flat uh NVLink giant GPU on the planet and is designed to take us to 10 trillion parameter models to enjoy when we get there next year. That's it for me and thank you for coming. I want to you to join us at OCP. We have lots of great talks. There's a couple here with AI tokenomics. We can explain in detail the work that's being done with NVIDIA in partnership with others on AI home vault and how the grid works and also our next generation DC architecture and how it's going to scale from today to tomorrow and beyond. And there are really cool display of all of the technology. If you want to see see how the connectors work with bus bars, all the things that we're contributing back to OCP and how we built Blackwell and Vera Rubin and moving forward some of the components of Vera Rubin Ultra there. You can see how we're actually contributing and helping the world build the factories that we're all producing together today. Thank you for coming. I hope you have enjoyed the rest of OCP and thank you. >> [applause]