Powering the Amazing Growth in AI Processing

October 4, 2018

Recently, a post on the OpenAI blog revealed that the amount of compute used in the largest AI training runs has been doubling every 3 ½ months since 2012. This research, conducted by Dario Amodei and Danny Hernandez, highlights the challenge of powering the highest-performance AI applications. In general-purpose supercomputing applications, the growth has been less dramatic, but power is already a major concern, with an annual table of the greenest systems getting almost as much attention as the list of the highest-performance installations.

One way to address this incredible exponential growth required by AI, which dwarfs Moore’s law, is the use of GPUs instead of conventional CPUs. At GTC 2018, Jensen Huang, president and CEO of nVidia, claimed that GPU-powered supercomputers incur 1/5 the cost, require 1/7 the space, and draw 1/7 the power when compared with conventional supercomputer racks.

Despite the impressive benefits of using GPUs, the reduction in power consumption by a factor of seven will be negated by the demand for more compute performance in high-end applications in just 10 months.

48V compared to conventional multiphase computing LinkedIn

The Need for a New Approach to Power in AI Applications

There have been many innovations in technology to help deliver the computational performance required by all AI applications, ranging from new architectures to high-performance interconnects. Data centers have moved from 12V to 48V power distribution within the racks, enabling much higher power levels, improved efficiency and greater power density. Powering the new devices, which draw huge currents at low voltages also required a fresh approach to power optimization.

Power-on-Package Technology

Power-on-Package MCM MCDIn August 2017, Vicor introduced Power-on-Package (PoP) at the Open Data Center Committee (ODCC) meeting in Beijing, China. Designed to enable the next generation of GPUs, CPUs and ASICs, PoP consists of Modular Current Multipliers (MCMs) that are mounted directly to the XPU substrate, together with Modular Current Multipliers (MCDs) that are mounted on the PCB.

The MCD drives MCMs at a low current, reducing interconnect losses by 10X and allowing 90% of the XPU pins previously required for power delivery to be reclaimed for expanded I/O functionality. The MCM multiplies the current and steps down the voltage, and as the current delivered by the MCMs does not traverse the XPU socket, losses are minimized between the MCM and the silicon.

Today, we offer an MCM that can deliver peak currents up to 1000A, and the technology has been recognized with industry awards.

Enabling nVidia GPUs

GTC 2018 saw nVidia announce the world’s most powerful AI system to date, the nVidia DGX2. Each system uses 16 SMX3 GPU cards that leverage Vicor Power-on-Package technology. Later that year, at GTC Taiwan, nVidia announced the HGX-2 cloud server platform, based on the same Vicor PoP technology as the DGX2.

Looking to the Future

It’s inevitable that AI will continue to demand higher computer performance, placing even greater challenges on power system designers. To keep up to date with the solutions Vicor develops to solve the problems faced by CPU, GPU and system designers, check the 48V system design and Power-on-Package resources on the Vicor website.


Tags: , ,

Comments are closed.

Find out more about our Cool-Power Buck Regulators subscribe to vicor newsletter Contact Us

Get Connected