Hardware Acceleration of Block-Diffusion LLM for Edge Devices
By Wei-Hsing Huang 1, Kiseok Lee 1, Ming-Yen Lee 1, Weiyu Sun 2, Cheng-Jhih Shih 2, Gayatri Tanksali 1, Arpit Khandelwal 2, Pin-Jun Chen 1, Yingyan Celine Lin 2, Shimeng Yu 1
1 School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA
2 School of Computer Science, Georgia Institute of Technology, Atlanta, USA
Abstract
Single-stream (batch-one) edge inference cannot amortize weight traffic across requests. Full-attention diffusion LLMs recompute the entire sequence at every step; native block diffusion makes completed blocks immutable and exactly cacheable, yet refinement still streams prefix KV and FFN weights. We co-design WIFiV-LPDDR, a wide-I/O LPDDR system for precision-tagged reads, BRQ-KV for a canonical low-rank-plus-INT8-residual prefix with query-dependent per-entry precision, and DAT-FFN for drift-mapped canonical replacement, adjacent-stage-corrected low-bit delta, or cached-state carry while keeping live activations unquantized. Both map to an input-stationary mixed-precision systolic array. For the evaluated 1.5B/7B models on modeled Jetson-class platforms, the full stack provides arithmetic-mean energy-reduction factors of 3.79x/3.96x and arithmetic-mean latency speedups of 2.88x/4.44x at the reported DAT-FFN settings; every corresponding compressed model-benchmark score drops by less than one absolute percentage point from its baseline.
Index Terms—Large language models, Diffusion models, Edge AI, Hardware acceleration.
To read the full article, click here
Related Semiconductor IP
- DFI 6.0 Verification IP
- TSMC 7nm 0V75 / 0V9 ESD Local Clamp – Low Cap
- TSMC 65nm 3V3 ESD Local Clamp – Rad Hard
- TSMC 5nm 1V8, 1.2V and 0.9V ESD Local Protection – Low Cap
- TSMC 3nm 3V3 ESD Local Clamp
Related Articles
- VitaLLM: A Versatile and Tiny Accelerator for Mixed-Precision LLM Inference on Edge Devices
- CD-PIM: A High-Bandwidth and Compute-Efficient LPDDR5-Based PIM for Low-Batch LLM Acceleration on Edge-Device
- MIPI in next generation of AI IoT devices at the edge
- Hardware Acceleration of Kolmogorov-Arnold Network (KAN) in Large-Scale Systems
Latest Articles
- Hardware Acceleration of Block-Diffusion LLM for Edge Devices
- Scalable AXI4 Transaction Monitoring for Mixed-Criticality SoCs: From Phase-Level Precision to ID-Level Efficiency
- Hardware-managed heterogeneous high-bandwidth memory and flash in LLM inference systems
- LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension
- A Process-Aware Hybrid Si/IGO Monolithic-3D 6T SRAM with BEOL Pass-Gates for the 2nm Node