Versat-AI: An ONNX-to-SoC Compiler for Model-Agnostic CGRA Edge Inference

By Rúben Teixeira ¹, João Barreiros Rodrigues ², Jaime Aguiar ^1,2, Jiao Li ³ and José T. de Sousa ^1,2
1 IObundle, Lda, Lisbon, Portugal
² INESC-ID; INESC INOV; Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal
³ Microelectronics Research Development Center, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200444, China

Abstract

Edge inference on resource-constrained embedded nodes demands accelerators that are energy-efficient and compact. This paper presents Versat-AI, an open-source compiler that accepts a standard Open Neural Network Exchange (ONNX) model and generates a complete, synthesisable RISC-V System-on- Chip (SoC) with an embedded CGRA accelerator. The key innovation is applying a known hardware merge strategy to collapse structurally compatible neural network operators into a physical CGRA instance. The Versat-AI compiler also derives memory-mapped interconnects, firmware drivers, and RISC-V application software co-generated by the Py2HWSW SoC framework, eliminating the manual hardware/software co-design effort that previously tied this accelerator’s own design lineage to a single target network. The next phase of the project is to extend this same automatic derivation from sizing the operator vocabulary to sizing per-operator parallel instancing and bandwidth, following the bandwidth-matched scaling principle already demonstrated, by hand, in this accelerator’s own design lineage. The current phase of the project has succeeded in creating an sound automation flow that produces an accelerator that maps each operator onto a single physical datapath instance and occupies 8,763 LUTs, 9,833 flip-flops, 4 DSPs, and 202 BRAMs on a Xilinx Kintex Ultrascale field-programmable gate array (FPGA)—a footprint unchanged across all evaluated models regardless of size—and draws 0.65 W (1.96 W for the complete SoC including DDR4 controller), achieving 2.3× to 9× speedup over a software-only baseline on four MLPerf Tiny benchmark tasks. The paper also surveys the two-decade lineage of reconfigurable accelerators that motivated Versat-AI’s design, and documents the open- source SoC platform development—including two prior integration attempts and the decisions that led to IOb-SoC.

Keywords: edge inference; CGRA; ONNX; RISC-V SoC; FPGA; hardware accelerator; deep learning; hardware/software co-generation; neural network; open-source; low power; energy efficiency

To read the full article, click here