ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design
By Zhongkai Yu 1, Chenyang Zhou 2, Yichen Lin 1, Hejia Zhang 1, Haotian Ye 1, Junxia Cui 1, Zaifeng Pan 1, Jishen Zhao 1, Yufei Ding 1
1 Department of Computer Science and Engineering, University of California San Diego, La Jolla, US
2 Columbia University, New York, US.

Abstract
While Large Language Models (LLMs) show significant potential in hardware engineering, current benchmarks suffer from saturation and limited task diversity, failing to reflect LLMs’ performance in real industrial workflows. To address this gap, we propose a comprehensive benchmark for AI-aided chip design that rigorously evaluates LLMs across three critical tasks: Verilog generation, debugging, and reference model generation. Our benchmark features 44 realistic modules with complex hierarchical struc tures, 89 systematic debugging cases, and 132 reference model samples across Python, SystemC, and CXXRTL. Evaluation results reveal substantial performance gaps, with state-of-the art Claude-4.5-opus achieving only 30.74% on Verilog generation and 13.33% on Python reference model generation, demonstrating significant challenges compared to existing saturated benchmarks where SOTA models achieve over 95% pass rates. Additionally, to help enhance LLMreference model generation, we provide an automated toolbox for high-quality training data generation, facilitating future research in this underexplored domain. Our code is available at https://github.com/zhongkaiyu/ChipBench.git.
To read the full article, click here
Related Semiconductor IP
- MACsec Engine optimized for 10Base-T1S/T1L applications
- Dual Port, Voltage Scaling Register File Compiler on GF22FDX+
- Single Port, Voltage Scaling, High Speed SRAM Memory Compiler on GF22FDX+
- I3C Full Controller / Target
- I3C HCI Host Controller
Related Articles
- TTP: A Hardware-Efficient Design for Precise Prefetching in Ray Tracing
- Integrating VESA DSC and MIPI DSI in a System-on-Chip (SoC): Addressing Design Challenges and Leveraging Arasan IP Portfolio
- The pivotal role power management IP plays in chip design
- ioPUF+: A PUF Based on I/O Pull-Up/Down Resistors for Secret Key Generation in IoT Nodes
Latest Articles
- An Agentic Python Framework for LLM-Driven UVM Testbench Synthesis with Closed-Loop Compiler Feedback
- Reconfiguring System-Level Protection Through Bounded Delegation in RISC-V MCUs
- SkipLane: Write-Once, Conflict-Free Accumulation for Zero-Skipping CNN Engines on FPGAs
- Priority-Aware Deficit Round Robin for AXI4 Bandwidth Fairness in ASAP-7nm Process
- Atik: A Shared-Mesh Transformer Accelerator for RISC-V