Optimizing power management in data centres amid surging AI and ML demands
Introduction
The relentless advance of AI, machine learning (ML), cryptocurrencies and cloud computing is dramatically reshaping data centres. Needing to support a wide array of services from high-resolution video streaming to complex AI-driven data processing that require intensive computational power, data centres are rapidly becoming one of the largest consumers of global energy resources. The International Energy Agency has projected that if current trends continue, these data centres could consume over 1,000 terawatt-hours by 2026, a stark increase from the 460 terawatt-hours recorded in 2022.
As data centres continue to grow and adapt to keep up with processing demands, they encounter significant challenges. In an era of unprecedented data throughput, power distribution strategies are evolving to ensure energy efficiency, reduced heat generation, optimised space utilisation, and effective cost management.
Traditionally, data centres were designed to handle a power supply of 30-40kW per cabinet. However, the advent of high-power CPUs and GPUs, such as Nvidia’s H100 AI accelerator which contains 80 billion transistors and has a thermal design power (TDP) of 700W, demands a re-evaluation of this standard. These components, characterised by their substantial thermal design power (TDP), push the boundaries of what conventional cooling and power systems can manage. As processing power continues to scale, the power requirements for a single cabinet will exceed 200kW.
Strategic Power Distribution Network (PDN) adjustments
To address these burgeoning power needs, significant adjustments to the power distribution network (PDN) are essential. The PDN must not only be robust but also adaptable to the dynamic changes in data centres. Some key considerations include:
Energy efficiency and cost management: Prioritising energy-efficient solutions helps save energy, reducing operational costs and environmental impact.
Voltage stability: High-density server settings require stable voltages despite high di/dt scenarios and step-load changes.
Noise and interference control: With the increase in switching frequencies, mitigating electromagnetic interference has become very important.
Advanced cooling solutions: As power density increases, traditional cooling methods become inadequate, prompting the need for innovative thermal management techniques.
Figure 1: Data centre power distribution network metrics and their direction of travel
Innovations in Intermediate Bus Architecture (IBA)
The implementation of Intermediate Bus Architecture (IBA) has become a cornerstone in modern data centre power design. IBA enables efficient energy conversion from higher to lower voltages, fundamental for supporting the latest high-performance processors.
For example, the BMR491 series is a fully regulated quarter brick from Flex Power Modules which exemplifies advanced DC/DC conversion technology. It has the capability to deliver up to 1,300W power while maintaining strict safety standards through high-voltage isolation.
As data centres evolve away from reliance on battery backups, the integration of non-isolated, board-mounted DC/DC converters has accelerated. These converters enable high power output without the need for external isolation in the same quarter brick format, thereby optimising both space and efficiency.
For example, the BMR351 delivers up to 1,600W of continuous power and peak power of 2,320W with an input voltage range of 40-60V, suitable for the 48V/54V applications found in data centres and a fully regulated output of 12V nominal makes it ideal to power downstream Point of Load (PoL) converters and integrated power stages such as the BMR510, along with powering circuitry that needs a tightly regulated output.
Figure 2: The intermediate bus architecture
Unregulated IBC adaptations
In the fast-paced environment of data centres, flexibility in voltage regulation is also paramount. Unregulated DC/DC converters provide this flexibility by allowing for conversion of input voltages to the necessary output voltages based on a ratiometric conversion. For instance, converting a 48V input with a ratio of 4:1 to a range of 10-15V output efficiently supports varied operational requirements without the need for complex regulation mechanisms.
The shift towards compact power solutions such as the BMR313 Intermediate Bus Converter (IBC) with a 4:1 input to output ratio, which delivers substantial power (1kW of continuous power and 3kW of peak power) in an ultra-small footprint, highlights an industry trend towards miniaturisation to maximise efficient use of on-board surface space. These advanced compact converters facilitate higher power outputs from significantly reduced sizes, enabling more power-dense configurations that are particularly advantageous in data centres where space is at a premium.
Figure 3: An arrangement with 4:1 IBC can yield 86.75% efficiency excluding connection resistance with a load of 360A at 0.8V supplied by multiple paralleled VRMs
The BMR313 series is an example of how current IBC technologies allow for the powering of additional system components or the paralleling of multiple VRM converters to support a larger single load. To accommodate the increased power output and associated heat generation, more advanced cooling solutions than standard forced air cooling can be necessary. Liquid cooling technologies, particularly Direct-to-Chip cooling – where the cooling fluid is delivered directly to the load via heatsinks with built-in piping – are becoming more prevalent to maintain system performance and reliability.
Vertical Power Delivery (VPD) systems
The optimisation of component layout plays a critical role in enhancing the efficiency of power systems. Vertical Power Delivery (VPD) represents a significant advancement in this area by typically arranging power components vertically beneath the load. This alignment minimises connection lengths and optimises thermal management through more efficient heat dissipation and reduced electromagnetic interference.
Figure 4: ‘Vertical Power Delivery’ for a processor
The challenges of component placement and cooling
Effective placement of components is crucial for optimising the thermal and electrical performance of data centres. Traditional horizontal layouts can impede cooling and restrict access to essential components. VPD systems overcome these challenges by ensuring that Voltage Regulator Modules (VRMs) are strategically positioned to facilitate optimal cooling and accessibility.
Conclusion
As data centres continue to grow both in size and functional complexity, the demand for innovative power management solutions becomes increasingly critical. These solutions are required to not only address the immediate operational requirements but also anticipate future needs and scalability. Adopting a holistic approach to power system design to incorporate advanced architectures like Intermediate Bus Architecture and innovative practices such as VPD, will be the key to ensuring that data centres can meet the high-efficiency thresholds required in today’s digital landscape. Additionally, Flex Power Designer software (available for free on our website) remains a vital tool in this process, offering ways to test and streamline design processes and enhance operational efficiency.
(first published in Components in Electronics)