video_OCP2026_Arista_Bechtolsheim_GW級資料中心_20261015

Yes, the gigawatt data center here. Thank you for waiting. Thank you. Thank you. Thank you. So, you have seen, you know, what's going on out there. Data centers that are the size of low lower town Manhattan. You know, unbelievable large designs on the gigawatt and multi- gigawatt era. And um you know those data centers obviously had a certain design parameter but if we started from scratch you know wouldn't it be nice if we could design a data center that is far smaller far cheaper to build far faster to build that can deliver better you know roundtrip latency and so on and has less environmental impact. So this is what this talk is about and this is a very important consideration that if you add it up over the next 10 years is worth hundreds of billions of dollars. So here's a rough projection of future installed capacity. This is not what's being planned. What's being planned is like double of that. But in terms of what may get built in the next 10 year, the US alone is 150 gawatt. Uh China another 85 gawatt. Europe is a little bit behind here. 15 gawatt. Worldwide 250 gawatt. You know these are large numbers. And and you have seen the pictures. This is uh stargate. Uh this is a picture of a place in um outside of Reno, Nevada, known as the Tahoe Reno Industrial Park. If you never heard of this, it's the largest industrial park in the world, 130,000 acres in the middle of a desert. There's like no tree growing. And they they had the ability to zone this industrial. So basically the Tesla Gigafactory there and the rest is data centers, uh power plants, you know, concrete plants. It's kind of a weird version of the future if you visit there, but there's plenty of room. People have thousands of acres of lots to build these large data centers. This was perhaps the largest data center that has been announced to date. Um I think that the marketing of saying it's as big as low Manhattan didn't really help because it sounds like data centers are going to use up the remaining, you know, land area for food. But um what basically what we have today is a power density like power per square feet which is around 500 watts per square feet or 5 kilowatts a square meter for those of you metric and you know this was optimized for the NVL 72 terrax and that's okay but next generation data sensor of course are much denser in particular the AI racks are going from you know 127 120 kilowatts to 600 800 megawatt size. Uh the switches also getting denser today in aircooled rack of switches is 30 kow with liquid cooling and XPO modules you can get to 6.5 pits a rack that would be between 120 and 250 kilowatt. So if you then look at this at the cluster level like an AI you know you you tie together 8 16 32 racks into a tightlyknit cluster u this is actually from maybe an Nvidia blueprint of data centers but there's 16 GPU racks another 16 network and storage racks and and liquid cooling units and this all adds up to 2.3 megawws that's the current standard that's what people are building today and physically looks like this so the the two of them together is what you just saw on the previous page. But that's not what you want to build going forward. You want to build something that's 10 times denser. And to illustrate this is this little footprint here uh is a 28.8 megawatt footprint, not 2.3 megawws in a size of 2700 square ft, which is a power density of 10 kilowatts per square feet, not 500 watts per square feet. Now this doesn't include transformers and you know BBUs and so on but this does include the compute the network the storage the CDUs specifically there's 32 GPU racks here at 800 kilowatts each that's 25 megawatt right there 16 networking racks at you know 125 kilows another 2 megawatt section storage racks at 6.5 another megawatt and 16 CDUs at 12.5 kilowatt it adds up exactly magically at 28.8 8. Now the power delivery for this is as big as the data hall. We need high high-end solid state transformers or SSTs. The latest and greatest of these will do 9.6 megawws in a unit that's size of a half container. You need four of these to get n plus1 redundant. You need an equivalent set of BBUs. And all of this will be wired up in a way we'll explain shortly to deliver N plus1 redundant power at that level. But tied together, you know, the effective power density is 5 kilowatts per square feet. And this assumes you're putting these very expensive transformers in the in a nicely shielded um concrete building. They could also be put outside. Now then you can stack these things. So to build a 230 megawatt data center, it's eight sets of what you just saw before. Like the spine is in the middle and you spread it out right and left. If you want to build the gigawatt type data center, it would be you know 32 of these pods total area 16 point 170,000 square feet which is roughly the area of 4 acres not thousand acres right and one thing I was going to point out here is that the maximum distance here using Manhattan type routing is 220 m which is a roundtrip latency of 2.1 microcond. If this was 10 times bigger that same roundtrip delay would be correspondingly larger. Now, so if you compare this to the numbers I just showed you earlier and this in in the orange line, it includes the power distribution, it is in fact 10 times denser. Now, you know, of course, you don't have to go to dens. You may want to have some spare space in this building, but there's an opportunity to radically improve density. And this has a number of advantages. So, one, the first one is there's just much fewer building materials. onetenth the lengths of fiber, copper, conduits, cooling pipes, concrete and steel and land area which does add up in including construction time meaning labor. The second one is you have less latency and the most important one I'm sorry is time to market. So it's there's no fixed number here but the rule of thumb is that from uh starting breaking ground not permitting phase but once you have the permit in place and you start digging it takes roughly two years to deliver a gigawatt type data center which is too long right nothing happens in the two years the world can change but you you can't turn it on until the data center is finished wouldn't it be great if that could be reduced to one year and the key way to reduce it to one year is to reduce the labor because the labor is there's not enough labor these days to do all these things faster. So it's a rough estimate and these are not scientific numbers is it cost roughly $10 billion to construct the shell the warm shell the cooling instruction electrical in not including the GPUs or the network just the outside components of a traditional data center uh and a lot of that 30 to 40 to 50% of that cost is actually labor if you build something on one10en that size my back of the envelope was you could say 40% of that or 4 billion per gawatt which adds up to 400 billion for the next 100 gigawatts in buildout. So this is not pocket change. Um now to build these things successfully the key is actually the power distribution and there we have because otherwise you're limited by the density of available transformer technologies and so on. So what I want to show you is some of the key components that are coming out. They're not necessarily available today. The first one is a solid state transformer at 9.6 megawws that takes 34.5 kW medium that's called medium voltage. what you get from the high voltage in and outputs 800 volts DC 98.7% efficient liquid cooled. Uh it's 6 m long, that's roughly 20 ft. So it's the size of a half container. Um and you know it's impressive, very dense design. You need four of these to do M plus one. Then we need hello the clicker here. We need some battery backup units. Um, a a good model that was recently announced in Europe is the CL Tenner Sithium BESS, which has six megawatt hours in the size of a 20ft container. And you can stack these and put them sort of back to back. Um, so the this was actually designed for like, you know, fast rapid discharge. The the discharge rate is 0.5. So what this means you can pull out three megawws over two hours and so to supply you know 28 megawatts or or really more than that for redundancy you would need a whole bunch of these but the key is that most power interruptions are less than two hours so this could actually negate the need for having generators on site which would be a positive for getting your permit faster um and then on the cooling side this is an existing product from a company called cool which is a 2 megawatt CDU in the space of a single rack so it's 750 m 1.6 1.2 deep very low power consumption and you know that's on the market today. So um to now get this redundant you have to basically provide redundancy all the way from the power grid all the way to your XPU mother card and what this means is every element on that path. So the yellow on the left is the 34.5 kilovolt medium voltage followed by the 800 volt you know in blue there is the BBU the solid state circuit breakers we'll talk about this in a second feeding directly to these racks on the rack there's a power converter not on the rack on the board from 18 volts to 50 volts you need four sets of these and then you have your you know 50 volt to 12 1/2vt converter everything from the left to that 50 volt point is fully redundant any one of these things can break and you won't notice And without that you know even though these individual components are very reliable there's so many of them you wouldn't want to take the risk. So the bottom line is um 800 volt is of course coming uh for sure and it has better efficiency than the traditional 40 volts. You save about three percentage points or so for a gigawatt that's 30 megawatt. So this adds up and then one other important component is the solid state circuit breaker. One problem with traditional circuit breakers is they take too long to actually disconnect the short and in that short time they can have what's known as an arc flash which depending on the energy you have can be catastrophic. In fact, it may be the single most dangerous thing in a data center could be one of these arc flashes. They can kill you. Um so to prevent arc flashes you have to reduce the time it takes a circuit breaker to break to to break. And traditional circuit breakers take tens of milliseconds. It's just not the answer. uh and with DC voltage you don't have the zero crossing which makes this even harder. So what you need is uh what's known as solidstate turkey breakers. They're in development by multiple companies. They can clear a fault in 10 microsconds and there's nothing that moves. There's no moving parts. It's an selium carbide magic chip that just turns it off. And this is fully digitally controlled. It also measures all the power and it's quote safe by design. You need one of these for everything that feeds that rack coming in to prevent, you know, any short circuit potential on that thing and everything upstream of course. So uh to sum this up on the I now call this the ultra dense u data center is you know lowering construction cost lowering time to market improving reducing speed of flight delays uh reducing environmental impact and having data centers that are more socially acceptable is a huge deal and you know this is for sure coming. Uh of course it's tied to high density megawatt class racks but this is the future. Now there's another topic that relates to this which is not the most pleasant topic to talk about which is uh as you may have read in the papers data centers have become targets for let's just call it for drone attacks. Uh this is a picture from the UAE in March 26 where not just one but three data centers in the same area were knocked out by drone attacks losing you know customer data in the process because it wasn't envisioned that all of them would go away at the same time. So uh and there's more since then right um so to make data centers quote resilient one has to look at the attack vector which unfortunately these days are drones and drones come in all kinds of sizes. This is not a funny topic. The one on the left is u the one that caused the damage on the first place. Uh it can carry today 50 kg but the new version can actually carry 100 kilogram of explosives and it's a kamicazi drone. It you know hits the target to explode. It can travel 2,500 kilometers. It's called the Poorman's cruise missile and it flies by, you know, GPS and finds its target. The one on the right is simply a DJI, you know, high-end consumer drone that can carry a two or three kilogram camera, but you could also strap some other things to the same drone. It will fly 15 kilometers over 30 minutes. Now, the the actual threat is the explosive. Of course, there's conventional explosive. to just create a huge blast. Then there's this thing called shaped charges or what's known as tank busters which have this very high velocity like we say liquid metal stream of material that comes out that can penetrate a 4ft concrete wall or 12 in of steel. It's unbelievable what these little things can do with two or three kilometers. Uh both of these normally are only available to call it military players. uh the ones on the right have been built by certain groups in the uh you know middle east on with using conventional tools but let's just say you know these available to government sort of sponsored actors now these two things represent two very distinct um uh dangers the high explosives of course have a tremendous over pressure which can crack walls and damage the infrastructure tremendously and it's not just cracking the wall when you have a concrete wall that's exposed to a high pressure uh shock wave, the inside of the wall just sort of explodes by itself even if it's not penetrated and you have these little particles that fly loose at sort of supersonic speeds. The shaped charges are a bit dangerous because if they make it into the in inside of any building, you know, they create a lot of problems and they they're very good at penetrating things. So basically to really design data centers for this unfortunate future where they're mission critical infrastructure and the enemy may decide that's what they want to target. Uh you have to look at both of these. Now ideally one could shoot the drones down but that's only allowed for the government like the military and they don't really guard data centers. You know they have trouble guarding their own airports. So um the the way that would work is you would have lighter and radars and optical cameras that track drones coming in. And there are ways to do what's called non-kinetic interception which is using very high powered acoustic systems that disrupt the ME stylescope in these drones and they get disoriented. But this I would say this is more stretched. The better strategy is to actually physically harden the building because that's what you're trying to protect. And the way you harden a building is pretty obvious. You need a multi-layer structure starting with a mesh or like steel stuff that's far away from the walls to capture the drone or to trigger the explosion before it gets ever close to the wall. So the multi-layer architecture here is high tensil steel meshes followed by hightensil fencing that it definitely triggers the explosion far away from the wall or the roof. And then the building itself should really have two layers of concrete that are separated with infill that slows down these you know shape charge kind of things. And it is possible to build a data center with these sort of design principles that will not have any damage on the inside no matter what this drone delivers to you. So to explain this in more detail here's these five layers that is basically you know that use different materials and they have to be sufficiently spaced apart. So this works best if you have extra area outside of the data center where you have a 50 foot distance from the met the steel mesh to the steel fence not a 50 foot to the outer wall then you need this filler gap enough space to divert it. So the uh let's call it key issue is actually the material used for building the building. If you use conventional concrete it will just break under a high pressure event. And there's this new material or there's a variety of new materials known as as a family hry performance concrete. The one that seems the most pre promising which is still kind of in the lab is using a combination of steel fibers and carbon nano fibers meshed together to have the highest flexual strength of any material and at the same time be impervious to water. So basically you have to mix up locally water the cement and the binder and the magic material to form this material. The benefit is it's so much stronger that you can actually build things with half the thickness. So instead of using 24 foot foundations, you can do 12 sorry 24in foundations, you can do with 12 in. Instead of a 12-in wall, you can build a 6-in wall with the same or superior characteristics. Um, you can also walk on these surfaces within like 48 hours. In normal concrete, you have to wait a week until they're strong enough. Um, and the thing is completely waterproof. I have to I'm told I have to go a little faster here. From a cost standpoint, they do cost a little bit more but perhaps less than what should assume because it's really on a per finished product basis, not on a material basis. You use much less material with this approach. So to sum this up here, this is this topic is not going away. You know, from the ocean, any data center in the US is within reach of one of these drones and the principles of design is well understood what needs to be done. Um the best materials are absolutely key and you know this allows people to sleep better at night or at least to get insurance for the data centers in a way that um that works. So final slide here um uh ultra high power highdensity data centers is the future. They're much cheaper to build. They're faster to build. They deliver less better performance. They're easier to fortify because they're much smaller and they have much less environmental impact. And this is obviously a fundamental change to the data center architecture where OCP and all the various activities going on here can be very helpful to get a common understanding of what one needs to do. And that's my last slide. Thank you very much.