video_OCP2026_Alibaba_ScaleUp光學從可插拔到NPO_20261015

Hello everyone. Good afternoon. And today I'm My name is Zhiping Yao. I'm senior director of Alibaba Group. Today I want to talk about one of the hardest problem in our infrastructure. How we scale? And why optics become a essential part of the scale up network? Uh here you see the title. My title have the MPO, right? The N stands for anything. It could be MPO, new package optics, could be uh UPO, ultra performance optics, could be IPO, you know, that's the new name we invented, right? It's a good name. IPO is we call it in the proposal package optics. I going to talk about this in this slide. It's a spectrum of optics, not a single solution. Okay, here's the the map. Uh it's You can see the distance dictate the network interconnect. On the very left side is the server. Inside the Inside the server we use the copper, right? The PCB to connect. And in the super node super node is like the Nvidia 72 the the big rack, right? That's super node. Inside super node is like a PCB or wire. The You see the Nvidia's Nvidia 72 those big cable cartridges that running on 200 gig series. And maybe I don't know, maybe it's 5,000 wires, right? It's a It's a very big, but this is fitting to the super node. It's called the scale up network. Then we move to the right is the pod size. Pod size roughly maybe dozens of racks or even hundreds of racks. The distance it's about you know, 50 m to 500 m. The optic become essential here. We use like a short reach optics or, you know, single mode or long reach optic the relatively like a DR4 or FR4 optics here. Um from pod to cluster, so it's like a, you know, 500 m or 2 km distance. Then all the way to to the campus level. Campus level become uh uh the DCI network or scale. We call it in AI domain we call it scale across network. So, here we use the zero optics, the DCI the DCI transport equipment for the the campus and also the metro. Uh look at the this gap. We use the Nvidia as an example, right? You can see a few years ago the NVOA the DGX box is all PCB PCB trace. The eight GPU will connect together through the high-speed PCB. Then a few years then the uh the NVI assembly rack the over and rack, it's used the cable cartridge. It's huge, very big, right? Everybody know this. And on the right side is a new NVR 576 they use the PCB technology from wire to the PCB technology to theoretically it can support up to 576, right? So, here is the all copper interconnect. Copper, of course, they have advantage in cost and power and also reliability, but their reach the reach is is extremely limited. It's roughly maybe maybe one a few meters, right? So, that's why, you know, in the future when the scale up keep going, the scale maybe go to like a thousands of TPUs, then thousands of accelerators in one domain, right? Then you hit the the copper wall. So, we need optics. So, why optics used for scale up? Because the first thing actually is expand the scale up demand from a single rack to multiple rack, maybe even more. And we think single layer scale up demand can support like a more than more than 1,000 GPU very soon, right? Today, it depends on the large radiator switch. Today, the switch we have is 512 port, but in the future next year, we will have like a 1,000 port switches. So, with a single tier switch, we can support up to, you know, 1,000 accelerators together under a single demand. The beauty of optics is that it decouple the computer node from the switch node. You know, in a in a rack in a AI rack, the switch and computer node are coupled together through the cable countries, but with the optic, we can easily decouple it. We can design and deliver the computer node or computer box and the switch box separately. That one give us the flexibility and high system availability and also the system resiliency >> [snorts] >> resiliency, that's right. Okay, so from technology point of view, high density, high bandwidth, and low power optical connect become an essential part for next generation of scale up. MPO is one of the key technologies here. Okay, the map. This is the Alibaba's actual performance network. We believe use optic technology to connect the switch node and the computer node relatively decoupled. And this can expand the domain, you know, from current the 72 GPU to 512 or even more if the new switch come out. This architecture, you can you can see from top, that's the switch node with the NPO inside and the computer node is also have a NPO. In the middle, there are some shuffle blocks to increase the radix of the of the whole whole scale network. Um The beauty of this design is like decoupling the computer node and switch node. And it it it will be very easy to deliver it in time and very quickly. On the bottom side is the switch. You can see this is single tier network. The most important piece of this network is the large radix switch. Um in you know, Tomahawk 5 is one of the the switch today and in the future right now the most cutting edge switch is Tomahawk 6, right? It's 100 T [snorts] switch with 512 port. In the future, next year or so we'll have a like a Tomahawk 7 with 200 T switch with 1024 port. The radix become larger and the larger. Then a single tier network become larger and the larger. We can easily connect 512 GPU together today and 1000 GPU future. Okay, this is the optic technology tree. The from left side is the traditional pluggable optics. I have already know it's a 4 FRO 4 return on optics. LRO linear return on optics and also LPO linear pluggable objects. On the right side you can see from top is the high density industry one that continue one to continue to use the pluggable objects. That is the I think that is the FRO LPO but with high density. XPO announced the last year, right? It's one of the pluggable high density pluggable form factor. And just yesterday I believe that Cisco and Nvidia announced the USFP. That's the new pluggable, you know, high density objects. That also continue the pluggable domain. On the middle is the MPO domain. MPO is the near package objects. The object engine sitting to the mainboard and connect to the switch via the mainboard PCB. And also we have the IPO. What is IPO? It's not an initial public offering, right? It's the we call it the in the poster package objects. So the object itself is sitting on the in poster use the elastomer connection to connect to the switch chip on top of the package. So the object engine can be removed separate from the switch. Right? It is very close to CPU but it can be separated. So it's kind of a middle between the MPO and the CPU. And the bottom side of course is the is the CPU is the most advanced and most compact solutions. >> [clears throat] >> This is a different view. It's like a package view of the high density high bandwidth objects. From left side is the pluggable objects. Everybody know the topology, right? It's in the middle it's like a MPO near package objects. The object engine is sitting to the mainboard and connect to the switch ASIC or XPUs through the member PCB. The IPO, just like I said, is called a interposer optics. Uh we rely on Broadcom's uh RCAS switch. So, there are some the RCA Thomas 6 uh the all the high-speed signal goes through the top side of the package through a LS Tuma connector connect to the RCA board. The RCA the optics on RCA we call it the IPO. On the right side is the CPU. CPU is co-packaged optics, right? Everybody know this. Uh this is uh it's the most advanced, the most compact, and uh most power efficient technology, but it also have a vendor lock-in, and also you know, very couple of them, right? Probably have a blast radius issue, and I also have a like a you know, manufacturer throughput issues. So, here is uh you know, form factor I go into a little bit of detail, right? I call it the MPO. M stands for everything. You can see from from the top side it's like a XPO or USFP which announced by Nvidia Cisco yesterday. It's pluggable, but high-density optics. Uh it it it because it's pluggable it's easy to to to field replaceable, right? You know, for the it's better for the maintenance, but I believe it's a signal integrity and system inside will be very challenging. It's it will be difficult to support a linear operations. The second uh is the MPO. Everybody know this. It's like a linear package. The main optic engine sit on the main board and connect to the switch ASIC like through the member PCB. The third one is the one probably I'm going to focus on is the IPO. So, the optic engine sit on the IPO board on top of uh the package. So, the signal integrity going to significantly improve over compared to the MPO. The last one is the CPU. CPU is is directly on the package, right? It's a It's a couple of solution, totally couple of solutions. Of course, it has the best performance, but like I said, it's you will not have a vendor freedom. You will be vendor locking. Okay, here's the technology side. You will there, you know, today 3.2T MPO uh will still use the OIF standard as like a 32 by 100 gig. Uh we already developed this and uh soon it will be, you know, production. The middle part of 6.4 MPO, that is the 32 by 200 gig versions, right? It's Right now, it's in a validation stage and uh you can see a lot of MSA say today, there was no not a single form factor here, right? So, like a CPX form factor is one of uh the form factor. And uh we uh have a several, you know, half scale in China. We have a MSA called uh UPO, ultra performance uh optics is supported like 6.4T on also the 12.8, you know, on a bottom. So, 12.8T is is is like a 32 by 400 gig. And here is still in the early R&D stage. We believe with the silicon photonics and also the TFIN hybrid bonding, right? We can support a 400 gig as like a 12.8T uh MPO and maybe I believe in the next year. Key technology comparison of all of a spectrum of the this MPO, right? And XPO pluggable we have the longest channels. The highest the loss in the system side. But there's a field repair serviceable and multi-source, right? It continue the current of pluggable success story. I believe it continue to be in the market, right? A lot of customer going to choose it. But challenge for this part is that the linear drive for especially for 200 gig will be difficult. 400 gigabit gig maybe impossible. Right, that's my my idea. So MPO is kind of in the middle middle field. It's a It's a number ball. The medium the channel loss is the channel distance is uh medium. You have a relatively lower loss compared to the pluggable. And it it could be replaceable in production. Right, but but not replaceable in the field. Still it support the multi-source, right? It's not a vendor lock-in. Then it can enable the ecosystem enable more vendors of the vendors to to work on a a common form factor, right? You enjoy the vendor probability here. And it's possible to you know what? I think it will be difficult to support a 400 gig, but it's could be possible. The IPO IPO is the one I going to focus on. On the On the interposer it has a shorter distance compared to the MPO. And could we believe from our simulation we could we think it could can be good up to 400 gig uh speed, right? So this is could be a good alternative to the CPU. CPU everybody know is the most advanced one, but it it the challenge for CPU is I think couple have a deep coupling between the OE and also ASIC. Right, maybe you have a the best ASIC vendors, but not a optic engine vendors. together, right? So, uh so we think that in the pulse optics it has very similar performance compared to the CPO, but it enjoy the vendor portability, it enjoy the multi-source. It It will enable the whole ecosystem. So, that's why, you know, I highlight this the IPO is the one we really want to, you know, pursue in next couple years. Okay, here's the most uh favorite part of uh this talk, right? Co-development of optic and ethics. It's not CPO, it is IPO. Uh we use the 100T the Thomas 6 RCA the switch. We We We power on this like a 1 month ago, you know, we pass traffic through all the channels. It's It's a great success, right? The the most innovative way here is uh the high-speed 200 gig 224 gig signals. All the 512 200 gig signals signals coming out of the top side of the switch package. Then we have the RCA card put on top of it. Together, you can see here it together with the with the elastomer uh contact, we test this we found out that the signal integrity is uh is super. It's much better than MPO. I'm not sure about compared to the CPO, it should be a little bit worse than CPO, but it's comparable. Right? So, so this one provide very low-risk migration from from, you know, the future uh 200 gig 200T switch to uh 400K switch in the future. So, like today the speed is 200 gig series, right? I believe I believe uh it will be easy migrate to the 400 gig series switch in the uh next year. Yeah, this switch has uh up and running for all. We are very excited about this. Okay, the last part rethinking the scale. So, I think it's three parts, right? So, copper remains important in short distance inside inside box inside the rack the copper still remain important, but objects become necessary as the scale grows. Right now it could go to like 500 well accelerated us in a single domain and could be like 1,000 in the future in very soon next year. So, MPO and IPO provide the most practical the ecosystem friendly path for scaling objects with broader vendor support. I believe that the IPO is very similar to CPU, but it will enable a broader broader ecosystem, right? The last part is the future scale up architecture should co-design with the compute with the optics and also with the network. That's all. Thank you very much.