Why You Should Create Your Own NPU Benchmarks
It Will Make for a Much Better NPU Vendor Evaluation.
We’ve written before about the ways benchmarks for NPUs can be manipulated to the point where you just can’t trust them. There are two common major gaps in collecting useful comparison data on NPU IP: [1] not specifically identifying the exact source code repository of a benchmark, and [2] not specifying that the entire benchmark code be run end-to-end, with any omissions reported in detail. Our blog explains these gaps.
However, there is a straight-forward, low-investment method to short-circuit all the vendor shenanigans and get a solid apples-to-apples result: Build Your Own Benchmarks. BYOB!
This might sound like a daunting task, but it isn’t. At the very beginning of your evaluation, it’s important to winnow the field of possible NPU vendors. This winnowing is essential now that a dozen or more IP companies are offering NPU “solutions.” At this stage, you don’t need to focus on absolute inference accuracy as much as you need to judge key metrics of [1] performance, [2] memory bandwidth, [3] breadth of NN model support; [4] breadth of NN operator support; and [5] speed and ease of porting of new networks using the vendors’ toolsets. Lovingly crafted quantization can come later.
Related Semiconductor IP
- NPU
- General Purpose Neural Processing Unit (NPU)
- NPU IP for Embedded AI
- NPU / AI accelerator with emphasis in LLM
- Highly scalable inference NPU IP for next-gen AI applications
Related Blogs
- Why You Can't Trust Your NPU Vendor's Benchmarks
- Why your DL accelerator should be replaced
- Should you make or buy your SoC connectivity IP? We interview Gerry Conlon, EVP Ensigma, Imagination
- Why you should care about Bluetooth LE Audio
Latest Blogs
- Why Choose Hard IP for Embedded FPGA in Aerospace and Defense Applications
- Migrating the CPU IP Development from MIPS to RISC-V Instruction Set Architecture
- Quintauris: Accelerating RISC-V Innovation for next-gen Hardware
- Say Goodbye to Limits and Hello to Freedom of Scalability in the MIPS P8700
- Why is Hard IP a Better Solution for Embedded FPGA (eFPGA) Technology?