Why You Can't Trust Your NPU Vendor's Benchmarks
Your Spreadsheet Doesn’t Tell the Whole Story.
Thinking of adding an NPU to your next SoC design? Then you’ll probably begin the search by sending prospective vendors a list of questions, typically called an RFI (Request for Information) or you may just send a Vendor Spreadsheet. These spreadsheets ask for information such as leadership team, IP design practices, financial status, production history, and – most importantly – performance information, aka “benchmarks”.
It's easy to get benchmark information on most IP – these benchmarks are well understood. For an analog I/O cell you might collect jitter specs. For a specific 128-pt complex FFT on a DSP there’s very little wiggle room for the vendor to shade the truth. However, it’s a real challenge for benchmarks for machine learning inference IP, which is usually called an NPU or NPU accelerator.
Why is it such a challenge for NPUs? There are two common major gaps in collecting useful “apples to apples” comparison data on NPU IP: [1] not specifically identifying the exact source code repository of a benchmark, and [2] not specifying that the entire benchmark code be run end to end, with any omissions reported in detail
To read the full article, click here
Related Semiconductor IP
- NPU
- Image Processing NPU IP
- Optional extension of NPX6 NPU tensor operations to include floating-point support with BF16 or BF16+FP16
- NPU IP for Data Center and Automotive
- General Purpose Neural Processing Unit (NPU)
Related Blogs
- Why You Should Create Your Own NPU Benchmarks
- Can You Rely Upon your NPU Vendor to be Your Customers' Data Science Team?
- Altera purchases optical network IP vendor Avalon Microelectronics
- Case Study: Choosing the Right Benchmarks for the Job
Latest Blogs
- The Growing Importance of PVT Monitoring for Silicon Lifecycle Management
- Unlock early software development for custom RISC-V designs with faster simulation
- HBM4 Boosts Memory Performance for AI Training
- Using AI to Accelerate Chip Design: Dynamic, Adaptive Flows
- Locking When Emulating Xtensa LX Multi-Core on a Xilinx FPGA