In Part I of this blog series, we explored the embodied and operational power trends of GPUs, observing how their power consumption has scaled over time. Specifically, we noted an approximate x1.4/year (CAGR of 41%) increase in power over the last 4 years, reflecting the growing computational demands and hardware complexity. However, power consumption alone doesn't tell the full story; to truly understand the evolution of GPUs, we must also examine their performance improvements and the resulting energy efficiency gains.
In this second part, we dive deeper into performance trends, particularly focusing on floating-point operations (FP32 and FP16), and assess how these advancements translate into performance-per-watt metrics. This analysis is especially important given the rapid acceleration of AI workloads, which rely heavily on GPUs for both training and inference.
Performance trends
GPUs traditionally rely on FP32 (single-precision floating-point) for most graphics, scientific computing, and general-purpose compute. However, the rise of deep learning has made FP16 (half-precision floating-point) increasingly important due to its faster computation and reduced memory bandwidth requirements. The newest accelerators push this further with dedicated tensor cores that run FP16 matrix operations many times faster than general-purpose units.
For this analysis, we evaluated 121 datacenter GPUs, excluding consumer-grade GPUs used in mobiles and PCs. The figures below show FP32, FP16 and FP16-tensor performance across release years.
Across 2020 to 2025, FP32 performance grew at roughly x1.55 per year (CAGR approximately 55%), FP16 at about x1.62 per year (approximately 62%), and FP16-tensor throughput fastest at around x1.70 per year (approximately 70%); a clear signal that the more AI-specific the workload, the faster the hardware has advanced.
Energy efficiency trends
Performance improvements are only part of the picture. To measure real progress, we need to understand how efficiently GPUs convert power into computational work, or their GFLOPs/watt.
The figures below show the FP32 and FP16 energy efficiency across release years, highlighting the most popular GPUs used for AI training.
Looking at FP32 energy efficiency: 2022 was an extreme outlier, with roughly 2.5x the efficiency of the previous year. Efficiency then fell sharply across 2023 (-25%) and 2024 (-34%), with a partial rebound in 2025 (on a small sample of three GPUs).
FP16 energy efficiency is steadier in magnitude than FP32 but changes direction more often year to year. Over the 2020 to 2025 window it roughly doubles (approximately x2.0), implying solid performance-per-watt gains on average for AI-training GPUs.
This trend suggests a strategic trade-off, where developers and manufacturers are prioritising raw speed and throughput, often associated with AI and machine learning tasks, over maintaining strict power efficiency targets.
The GFLOPs/watt metric provides a theoretical peak of raw computational energy efficiency. However, in real-world scenarios, particularly for AI, other benchmarks offer a more practical measure. Notably, standardised AI performance suites like MLPerf Training measure the end-to-end efficiency on typical AI tasks, capturing the total system energy drain from memory, interconnects, and other components, providing a more complete picture of real-world energy efficiency.
Unfortunately, the published MLPerf Training results are still too sparse to construct a robust long-term trend. However, notable patterns emerge, especially for models like Llama 2 70B LoRA.
Conclusion
To summarise:
- Over the last four years, GPUs perform roughly x1.55 (FP32) to x1.62 (FP16), and up to x1.70 for FP16-tensor, more work each year than their predecessors.
- However, they also consume about x1.4 more power each year.
- The net effect on energy efficiency has been mixed, sometimes positive and sometimes negative:
- FP32 efficiency: a spike in 2022, followed by a sustained decline in 2023 and 2024, with a partial rebound in 2025 (on a small sample of three GPUs).
- FP16 efficiency: choppy, but on a generally upward trajectory.
- MLPerf Llama 2 70B LoRA efficiency: with the exception of GB200 and GB300, real-world efficiency declines across newer GPU generations.
Meanwhile, AI workloads, particularly deep learning training and inference, are estimated to be growing at an exponential rate. This explosive demand means that even with continuous efficiency improvements, GPU power consumption and overall system energy demands will continue to rise sharply.
In essence, the current rate of GPU efficiency improvements may still struggle to keep pace with the unprecedented growth of AI computational needs. This underscores the importance of continued innovation in hardware architecture, software and model optimisations, as well as alternative approaches to traditional GPUs, such as custom silicon, specialised AI accelerators and other ASIC-based solutions.
For example, Google's Tensor Processing Units (TPUs) are designed to handle deep learning workloads. Unlike general-purpose GPUs, their special features, such as the matrix multiply unit (MXU) and proprietary interconnect topology, make them ideal for running AI training and inference. Google has begun mass-deploying its seventh-generation Ironwood TPU (TPU v7), offering up to 4,614 TFLOPS of peak performance and dramatically greater energy efficiency compared with earlier TPU generations.