Su姿丰: AMD is expected to improve energy efficiency by 100 times before 2027

2024-04-05

Pursuing lower energy consumption and higher performance.

AMD CEO Dr. Lisa Su attended the imec ITF World 2024 conference and received the imec Innovation and Industry Leadership Award, joining other recipients such as Gordon Moore, Morris Chang, and Bill Gates. After accepting the award, Dr. Su began her keynote speech by outlining the steps AMD has taken to achieve the company's 30x25 goal, which aims to increase the energy efficiency of computing nodes by 30 times by 2025. Dr. Su announced that AMD is not only on track to meet this target but now also sees a path to more than 100 times improvement by 2026-2027.

The explosive growth of generative AI Large Language Models (LLMs) like ChatGPT has brought concerns about AI power usage to the forefront, but AMD anticipated the high energy demands of AI as early as 2021. At that time, AMD began focusing on the 30x25 goal to improve the energy efficiency of data center computing nodes, particularly considering the power consumption of AI and HPC as an imminent issue. AMD set its first energy target in 2014 with its initial 25x20 goal, which aimed to increase the energy efficiency of consumer processors by 25 times by 2020.

Advertisement

This issue is now at the forefront. As the world's largest companies vie for dominance in artificial intelligence, generative AI is driving rapid expansion of data centers. However, public power grids are not prepared for the sudden surge in power-hungry data centers, making electricity a new limiting factor. Due to the constraints of grid capacity, infrastructure, and environmental issues, the available power for new and expanded data centers is strictly limited. In fact, many new data centers are being built next to power plants to ensure a power supply, and the overwhelming demand has even reignited the push for small modular reactors (SMRs) to supply various data centers.

The problem will only intensify as the computational requirements for training models increase. Dr. Su pointed out that the scale of the first image and voice recognition AI models doubled every two years, which is roughly in line with the pace of progress in computing power over the past decade. However, the scale of generative AI models is now growing at a rate of 20 times per year, outpacing the progress of computing and memory. Dr. Su stated that while today's largest models are trained on tens of thousands of GPUs, consuming tens of thousands of megawatt-hours, the rapidly expanding model sizes may soon require up to hundreds of thousands of GPUs for training, potentially requiring several thousand megawatts of power to train a single model. This is clearly unsustainable.

AMD has a multi-pronged strategy to improve energy efficiency, including a broad approach that extends from its chip architecture and advanced packaging strategies to AI-specific architectures, system and data center-level tuning, and software and hardware co-design initiatives.

Of course, silicon is the cornerstone. Dr. Su noted that the 3nm full-gate-all-around (GAA) transistors are the next step for AMD on the chip roadmap to improve power efficiency and performance while continuing to focus on advanced packaging and interconnects for more energy-efficient and cost-effective modular designs. Advanced packaging plays a key role in scaling designs to deliver greater horsepower within the constraints of a single chip package, with AMD employing a mix of 2.5D and 3D packaging to maximize the compute per watt per square millimeter of data center chips.Due to the long distances involved, transferring data between server nodes and server racks consumes additional power, so optimizing data locality can save a significant amount of power consumption. AMD's MI300X is a great example of the efficiency brought by manufacturing increasingly larger chip packages—this chip has 153 billion transistors distributed across 12 smaller chips, paired with 24 HBM3 chips, providing a memory capacity of 192GB, all of which can be provided to the GPU as local memory. Combined with the power consumption and performance optimization between units within the package, the extremely high computational and memory density brings more data closer to the processing cores, thereby reducing the energy required to transfer data.

Li Su-zhi said that while hardware optimization is important, AMD has also achieved impressive results in the collaborative optimization of hardware and software. Using low-precision numerical formats can improve power efficiency and performance, so designing under specific hardware acceleration is very important for continuous scaling. As you saw in the slides above, switching to low-precision formats such as FP4 will greatly increase the FLOPs consumed per joule of energy—compared with FP32, the energy efficiency of FP8 is 15 times higher, and the power efficiency of FP4 is about 30 times higher.

Lower precision will lead to lower accuracy, but Li Su-zhi emphasized that advanced quantization technology helps to solve this problem. In fact, even MXFP6 can produce similar accuracy to FP32, and only on MXFP4 can some models see a decline, while their models are still equally accurate. Work on improving the accuracy of low-precision formats is ongoing, so we may even see MXFP4 become as accurate as FP32 in more models in the future.

Overall, Li Su-zhi said that AMD has surpassed the industry's progress in terms of power efficiency per node, as the company is still striving to achieve its 30-fold improvement in power efficiency. Li Su-zhi expects this trend to continue, she said: "Based on what we see today, through this type of innovation, we believe we can do better. By 2026 and 2027, we expect to complete more than 100 times the work. There is still a lot, a lot more we can do in this field."

Imec's activities attracted speakers from various semiconductor companies such as ASML, TSMC, Intel, and SK Hynix, and many speakers agreed with Li Su-zhi's view that data center power consumption and its associated environmental consequences are becoming a primary issue.

Li Su-zhi pointed out that continuing the pace of improving energy efficiency requires the efforts of the entire industry. "We have the opportunity to drive this ecosystem by combining many different capabilities and many different expertise. I think this is the key to the next generation of innovation."

"We are all experts in certain fields, but when we bring together process personnel, packaging personnel, hardware designers, architects, software models, and system designers, we can achieve a truly bending the future innovation curve with overall design capabilities," said Li Su-zhi.

Leave a comment

Submit