Why You Can't Trust Your NPU Vendor's Benchmarks
Your Spreadsheet Doesn’t Tell the Whole Story.
Thinking of adding an NPU to your next SoC design? Then you’ll probably begin the search by sending prospective vendors a list of questions, typically called an RFI (Request for Information) or you may just send a Vendor Spreadsheet. These spreadsheets ask for information such as leadership team, IP design practices, financial status, production history, and – most importantly – performance information, aka “benchmarks”.
It's easy to get benchmark information on most IP – these benchmarks are well understood. For an analog I/O cell you might collect jitter specs. For a specific 128-pt complex FFT on a DSP there’s very little wiggle room for the vendor to shade the truth. However, it’s a real challenge for benchmarks for machine learning inference IP, which is usually called an NPU or NPU accelerator.
Why is it such a challenge for NPUs? There are two common major gaps in collecting useful “apples to apples” comparison data on NPU IP: [1] not specifically identifying the exact source code repository of a benchmark, and [2] not specifying that the entire benchmark code be run end to end, with any omissions reported in detail
Related Semiconductor IP
- NPU
- General Purpose Neural Processing Unit (NPU)
- NPU IP for Embedded AI
- NPU / AI accelerator with emphasis in LLM
- Highly scalable inference NPU IP for next-gen AI applications
Related Blogs
- Why You Should Create Your Own NPU Benchmarks
- Can You Rely Upon your NPU Vendor to be Your Customers' Data Science Team?
- Altera purchases optical network IP vendor Avalon Microelectronics
- FPGA design tool vendor GateRocket has ceased operations
Latest Blogs
- Why Choose Hard IP for Embedded FPGA in Aerospace and Defense Applications
- Migrating the CPU IP Development from MIPS to RISC-V Instruction Set Architecture
- Quintauris: Accelerating RISC-V Innovation for next-gen Hardware
- Say Goodbye to Limits and Hello to Freedom of Scalability in the MIPS P8700
- Why is Hard IP a Better Solution for Embedded FPGA (eFPGA) Technology?