Hardware Acceleration of Block-Diffusion LLM for Edge Devices

By Wei-Hsing Huang ¹, Kiseok Lee ¹, Ming-Yen Lee ¹, Weiyu Sun ², Cheng-Jhih Shih ², Gayatri Tanksali ¹, Arpit Khandelwal ², Pin-Jun Chen ¹, Yingyan Celine Lin ², Shimeng Yu ^1
1 School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA
² School of Computer Science, Georgia Institute of Technology, Atlanta, USA

Abstract

Single-stream (batch-one) edge inference cannot amortize weight traffic across requests. Full-attention diffusion LLMs recompute the entire sequence at every step; native block diffusion makes completed blocks immutable and exactly cacheable, yet refinement still streams prefix KV and FFN weights. We co-design WIFiV-LPDDR, a wide-I/O LPDDR system for precision-tagged reads, BRQ-KV for a canonical low-rank-plus-INT8-residual prefix with query-dependent per-entry precision, and DAT-FFN for drift-mapped canonical replacement, adjacent-stage-corrected low-bit delta, or cached-state carry while keeping live activations unquantized. Both map to an input-stationary mixed-precision systolic array. For the evaluated 1.5B/7B models on modeled Jetson-class platforms, the full stack provides arithmetic-mean energy-reduction factors of 3.79x/3.96x and arithmetic-mean latency speedups of 2.88x/4.44x at the reported DAT-FFN settings; every corresponding compressed model-benchmark score drops by less than one absolute percentage point from its baseline.

Index Terms—Large language models, Diffusion models, Edge AI, Hardware acceleration.

To read the full article, click here