Why Software is Critical for AI Inference Accelerators
By Geoff Tate, Flex Logix (April 23, 2020)
Inference accelerators represent an incredible market opportunity not only to chip and IP companies, but also to the customers who desperately need them. As inference accelerators come to market, a common comment we hear is: “Why is my inference chip not performing like it was designed to?”
Oftentimes, the simple answer is the software.
Software is key
All inference accelerators today are programmable because customers believe their model will evolve over time. This programmability will allow them to take advantage of enhancements in the future, something that would not be possible with hard-wired accelerators. However, customers want this programmability in a way where they can get the most throughput for a certain cost, and for a certain amount of power. This means they have to use the hardware very efficiently. The only way to do this is to design the software in parallel with the hardware to make sure they work together very well to achieve the maximum throughput.
Related Semiconductor IP
- eFPGA
- Radiation-Hardened eFPGA
- eFPGA IP as a synthesizable RTL core
- eFPGA IP and FPGA Software Built on GLOBALFOUNDRIES 22FDX
- eFPGA IP and FPGA Software Built on Samsung Foundry 28nm FDSOI
Related White Papers
- Top 5 Reasons why CPU is the Best Processor for AI Inference
- Why Interlaken is a great choice for architecting chip to chip communications in AI chips
- Why the Memory Subsystem is Critical in Inferencing Chips
- AI Edge Inference is Totally Different to Data Center
Latest White Papers
- How silicon and circuit optimizations help FPGAs offer lower size, power and cost in video bridging applications
- Sustainable Hardware Specialization
- PCIe IP With Enhanced Security For The Automotive Market
- Top 5 Reasons why CPU is the Best Processor for AI Inference
- CANsec: Security for the Third Generation of the CAN Bus