Partitioning to optimize AI inference for multi-core platforms
By Rami Drucker, Ceva
EDN (January 8, 2024)
Not so long ago, artificial intelligence (AI) inference at the edge was a novelty easily supported by a single neural processing unit (NPU) IP accelerator embedded in the edge device. Expectations have accelerated rapidly since then. Now we want embedded AI inference to handle multiple cameras, complex scene segmentation, voice recognition with intelligent noise suppression, fusion between multiple sensors, and now very large and complex generative AI models.
Such applications can deliver acceptable throughput for edge products only when run on multi-core AI processors. NPU IP accelerators are already available to meet this need, extending to eight or more parallel cores and able to handle multiple inference tasks in parallel. But how should you partition expected AI inference workloads for your product to take maximum advantage of all that horsepower?
To read the full article, click here
Related Semiconductor IP
- TSMC 7nm 0V75 / 0V9 ESD Local Clamp – Low Cap
- TSMC 65nm 3V3 ESD Local Clamp – Rad Hard
- TSMC 5nm 1V8, 1.2V and 0.9V ESD Local Protection – Low Cap
- TSMC 3nm 3V3 ESD Local Clamp
- TSMC 3nm 1V2 ESD Local Clamp – Low Capacitance
Related Articles
- MultiVic: A Time-Predictable RISC-V Multi-Core Processor Optimized for Neural Network Inference
- AIA: A 16nm Multicore SoC for Approximate Inference Acceleration Exploiting Non-normalized Knuth-Yao Sampling and Inter-Core Register Sharing
- AI Edge Inference is Totally Different to Data Center
- Making Strong Error-Correcting Codes Work Effectively for HBM in AI Inference
Latest Articles
- LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension
- A Process-Aware Hybrid Si/IGO Monolithic-3D 6T SRAM with BEOL Pass-Gates for the 2nm Node
- Automated Estimation of MBIST Area and Test Time in Heterogeneous Memory IPs via Stacked Ensemble Framework
- VIPER: Architecture-Aware Performance Modeling for Processing-in-Memory Design-Space Exploration
- CTTE: An Open Dual-Protocol RISC-V Trace Encoder for N-Trace and E-Trace