ReGate: Enabling Power Gating in Neural Processing Units
By Yuqi Xue, Jian Huang (University of Illinois Urbana-Champaign)
Abstract
The energy efficiency of neural processing units (NPU) is playing a critical role in developing sustainable data centers. Our study with different generations of NPU chips reveals that 30%-72% of their energy consumption is contributed by static power dissipation, due to the lack of power management support in modern NPU chips. In this paper, we present ReGate, which enables fine-grained power-gating of each hardware component in NPU chips with hardware/software co-design. Unlike conventional power-gating techniques for generic processors, enabling power-gating in NPUs faces unique challenges due to the fundamental difference in hardware architecture and program execution model. To address these challenges, we carefully investigate the power-gating opportunities in each component of NPU chips and decide the best-fit power management scheme (i.e., hardware- vs. software-managed power gating). Specifically, for systolic arrays (SAs) that have deterministic execution patterns, ReGate enables cycle-level power gating at the granularity of processing elements (PEs) following the inherent dataflow execution in SAs. For inter-chip interconnect (ICI) and HBM controllers that have long idle intervals, ReGate employs a lightweight hardware-based idle-detection mechanism. For vector units and SRAM whose idle periods vary significantly depending on workload patterns, ReGate extends the NPU ISA and allows software like compilers to manage the power gating. With implementation on a production-level NPU simulator, we show that ReGate can reduce the energy consumption of NPU chips by up to 32.8% (15.5% on average), with negligible impact on AI workload performance. The hardware implementation of power-gating logic introduces less than 3.3% overhead in NPU chips.
To read the full article, click here
Related Semiconductor IP
- NPU
- NPU IP Core for Mobile
- NPU IP Core for Edge
- Specialized Video Processing NPU IP
- NPU IP Core for Data Center
Related White Papers
- New generation RISC processing power - Green technology engenders new business opportunities
- Sequential clock gating maximizes power savings at IP level
- Context Based Clock Gating Technique For Low Power Designs of IoT Applications - A DesignWare IP Case Study
- Achieving Low power with Active Clock Gating for IoT in IPs
Latest White Papers
- ReGate: Enabling Power Gating in Neural Processing Units
- Morphlux: Programmable chip-to-chip photonic fabrics in multi-accelerator servers for ML
- RISC-V basics: The truth about custom extensions
- Unlocking the Power of Digital Twins in ASICs with Adaptable eFPGA Hardware
- Security Enclave Architecture for Heterogeneous Security Primitives for Supply-Chain Attacks