Nvidia might be pushing
its customers towards adopting 800 volt power in their datacenters, but Schneider
Electric executives don’t expect more than 10 percent of new AI nodes to move to
this high voltage anytime soon.
Companies are already
packing in racks at 140 kilowatts and increasingly at 200 kilowatts. But Nvidia’s
“Kyber” rack architecture, which envisages the fabled 1 MW rack, is predicated
on 800 volts.
At a datacenter event
in Buffalo, New York, Schneider Electric’s cloud and service provider technical
and solutions director Rob Bunger explained that conventional in-rack power is
at 48 volts, which is “touch safe.” But mechanical/electrical issues start
kicking in as racks head to 400 kilowatts, Bunger continued. This includes
congestion in the rack from feeds and converters and the power capability of
the rack of the bus bar at low voltage.
As Bunger explained, a
150 kilowatt rack running 72 GPUs would need eight “whips” or power cables. A 1,000
kilowatts rack running 144 GPUs would need 32 whips – and they would be much
larger. “Which is impractical. It
doesn’t make any sense at all,” says Bunger.
So, Bunger continued,
“The two things you want to do is one, move those converters –rectifiers turning
AC into DC or stepping DC voltages – out of the rack. You consolidate them and
you up the voltage to get more power per wire.”
Getting to 800 volts will
initially be through sidecars in the rack itself, he said, but that will itself
use up rack space.
“It starts to become a
little bit impractical to do a ton of side cars as the datacenter gets larger
and larger, so there will be centralized solutions that allow you to do larger
chunks of power conversion to do multiple routes, call it centralized DC distribution.”
But he said, “Now you
have to think about, ‘Okay, I have potentially the conversion happening outside
of the white space, and I have to get that 800 volts to the IT rack,’ so that
you have the power distribution and protection and the sensitivity, and all
this stuff.”
He predicted small
scale deployments by the end of this year, with “more centralized” power
distribution solutions by 2029. Initial sidecar solutions would “be able to
provide 600 kilowatts to 1 megawatt worth of power, so you can imagine that’s
going to be being able to power one, two, or three of these very high-end AI
racks.”
For the centralized solutions,
“We’ll have converters that’ll be able to take AC to 800 volts, and these will
come in chunk sizes of about two to five megawatts, roughly, and that’s what we
see in the middle.”
“You see the term DC
UPS,” he added, “So there would be energy storage batteries coupled with these.
So from a perspective of uninterrupted power of DC output, it would be
considered the DC UPS. The sidecar again can be configured with energy
storage.”
But while Nvidia is
ploughing ahead with 800 volts, Bunger said this doesn’t mean large scale
retrofitting of existing sites. “We think by 2030 maybe 10 percent of the new
AI nodes coming out will be required to have 800 volt DC.”
And even brand new datacenters
will not be entirely based on 800 volts. “We’ll likely see pockets of it within
datacenters,” Bunger said, together with a mix of medium to low voltage.
Nevertheless, he said,
this would still have an impact on overall datacenter design as density
increases. “The ratio of facility equipment to IT equipment is definitely
changing.”
Unsurprisingly,
Schneider said a holistic approach to reworking power infrastructure made sense
– or a single vendor to put it another way. “You really have to think of the
holistic solution from the panel boards, the bus waves, the circuit breakers.”
“So, the first phase
of seeing 800 volts getting deployed will be a very highly engineered full
system design to make sure it’s very reliable. As the whole market matures,
both from a standards perspective, and more options, we will start to see more
interoperability between products.”
Things are further
complicated by the nature of AI workloads compared to traditional cloud workloads,
Bunger added.
Traditionally, “The
whole purpose of the facility was to protect the IT load, you know, high nine,
high safety, and high nines of availability”
AI workloads, by
comparison, are not static. “Especially with AI training. They pulse, and so
you have to think about what that effect is on the grid, pulsing load, and then
very, very large loads.”
Fault ride through
becomes an issue at this point. “So not
only does the datacenter facility have to protect the IT load historically, it
now also have to protect the grid at the same time.”
Schneider chief
marketing officer Kevin Brown added that there was nothing particularly magical
about 800 volts, “The reason the industry is going to 800 volt DC is because of
EVs. Because there’s a supply chain available to deliver connectors, and
there’s an understanding of how it works, and so forth.”
If changing power architecture
wasn’t enough to keep AI and HPC engineers busy, Schneider Electric also argued
that the shift to liquid cooling as these power hungry racks proliferate will
help AI datacenters flush out more tokens. And the company pushed back on
claims that AI datacenters are causing a water crisis.
Tuan Hoang,
Schneider’s head of cooling technology and product development, outlined a case
study of water consumption by putative AI datacenters in Texas and Paris based
on the company’s reference designs, comparing the impact of evaporative cooling
versus liquid cooled systems.
The study compared the
impact of running a 100 megawatt datacenter in Texas and Paris.
Based on these, he
said, a 100 megawatt air-cooled/evaporative tower site in Dallas would deliver
a 1.148 PUE and consume 161 Olympic swimming pools of water, while delivering
1.99×108 tokens. Shift that to more temperate Paris, and the PUE
comes in at 1.11, with the same water consumption but 2.78×108
tokens delivered.
Here is the
comparative data for the Dallas scenario:
And here it is for the
Paris scenario:
And finally, here is a
summary table comparing the two:
A datacenter in Texas,
using liquid cooling with Vera-Rubin AI systems from Nvidia, would deliver a
1.04 PUE, consume 79 pools of water, and generate 2.52×1011 tokens.
Its Paris-based equivalent would deliver 1.04 PUE, consume 20 pools of water,
and generate 2.91×1011 tokens.
Just for comparison,
an Olympic pool equates to 2,500 m3 of water, while a typical Dallas
household consumes 416 m3 of water a year, and a Paris household
sloshes through 110 m3 a year.
Of course, the real
challenge is getting permits to squeeze in an AI scale datacenter in the compact,
power constrained City of Lights compared to power and land abundant Texas.
But Huang said the
exercise was significant give that water consumption is one of the reasons
communities are pushing back against datacenters. He said the numbers showed
that “water consumption is a choice, not a requirement.”
Using water to extract
heat more efficiently comes at a cost, he said. “But the location decision also
changes the amount of water that’s consumed. Imagine extrapolating that this
out to the extreme to go to the northern or cooler environment, or you go to
the Middle East, or desert environment.”
And Schneider CMO Brown
added, the industry had to do a better job explaining that liquid cooling “does
not mean water use.” Rather, water use is down to the use of evaporative
cooling, which was adopted to get better energy efficiency. But AI datacenters
are effectively synonymous with liquid cooling.
By adopting liquid
cooling, Brown said “because the temperatures change, you have unique
opportunities with that architectural change, our water use goes down
nominally, relatively speaking. The amount of kilowatts you’re consuming and
tokens you’re generating will become much more efficient.”
The myth that AI data
centers, because they’re liquid cooled, are consuming water, is “not true, you
can avoid it and still have that efficient data center.”
Though local communities are likely to have
multiple other objections to having a 200 acre plus AI facility dropped on
them.
