At the top of the hype cycle

Nov 27, 2023

As the hype around Generative AI hits fever pitch across the data center industry, companies are scrambling to plan, prepare and assess the optimal ways to leverage the proliferating technology. From the arms race between the big hyperscalers to secure as many GPUs as possible, to widespread proofs of concepts being executed across the industry, Generative AI is vying to become the application that revolutionizes the world.

Whether or not that lofty goal materializes remains to be seen. After all, the last time the technology industry experienced such a heightened level of exuberance was around the first internet boom in the late 90’s ― and we all remember how that ended. In fact, in its latest Infrastructure Strategy 2023 Hype Cycle, Gartner awarded Generative AI the top spot for inflated expectations.

Regardless, we have reached a point where virtually every industry discussion is rotating around the technology topic, with market players being deemed irrelevant if they don’t mention AI in their communication and financial analysts counting the number of times the term is mentioned in earnings calls. Whether it’s ultimately deemed indispensable or hype, this technology is top-of-mind among today’s data center operators.

However, at the moment, the number of deployments is lacking, in large part due to the numerous barriers for adoption, including data confidentiality, data security and shortage of skilled labor. Many companies would prefer to run these systems on-premise in order to avoid recurring costs such as those spurred by cloud computing. Yet this likely will not be feasible at least in the near term, considering the inability for organizations to obtain needed GPUs and the lack of competencies to build these highly sophisticated models.

A primary driver of the high cost of Generative AI systems is the massive amount of power they require, forcing incredible power densities at the rack level. Compared to the traditional 5-10 kW needed for general purpose IT applications, these systems can demand a whopping 80-100 kW per rack. Existing data centers have not been designed to accommodate this level of power, nor the resulting requirements for cooling and water consumption.

Further complicating the challenge is the quest to maintain sustainable operations amid drastic technology changes. Data center operators will undoubtedly be exploring solutions such as replacing generator sets with outdoor batteries for backup power, the ability to interact with the grid and liquid cooling to reduce the whopping energy bill associated with cooling.

As a result, this will create massive demand in the coming years for infrastructure providers like Eaton to help build this next generation of data centers. Or multiple generations, as the case may be, considering that the requirements vary significantly between sites designed for LLM training or inference. We are currently discussing 400 MW sites ― representing 10 times the power requirements of a typical hyperscale data center, which was already considered massive just a few months ago.

The jury is still out for inference sites, which may be smaller than 40 MW as companies in major metro areas attempt to strike a balance between the need for low latency and rising power availability challenges. In addition, because all sites will not be fully dedicated to AI applications, there will likely be a lot of hybrid deployments.

The fact is, a scarce number of companies are equipped to tackle the numerous challenges and abundance of issues demanded by the emerging technology ― and even fewer are capable of providing a complete solution that not only meets the needs of customers, but will deliver sustainable outcomes for years to come. Eaton is one. Because our comprehensive range of solutions and services address power requirements spanning from the utility substation down to the rack PDU powering the technology’s GPU-laden servers, we have already seen pent-up demand for our products, as well as countless inquiries from customers trying to proactively plan for the paradigm shift.

Regardless of Generative AI’s ultimate success or endurance, the many considerations around the technology have catapulted power to a chief concern among data center operators. From determining how to optimize existing designs and operate more efficiently, to selecting the best components and architect them to achieve positive financial outcomes, to ensuring uptime at a feasible cost, data center professionals will undoubtedly have much to sort through as AI technology remains at the top of the hype cycle.