The role of cache in AI processor design
By Frank Schirrmeister, Arteris
EDN (March 22, 2024)
Artificial intelligence (AI) is making its presence felt everywhere these days, from the data centers at the Internet’s core to sensors and handheld devices like smartphones at the Internet’s edge and every point in between, such as autonomous robots and vehicles. For the purposes of this article, we recognize the term AI to embrace machine learning and deep learning.
There are two main aspects to AI: training, which is predominantly performed in data centers, and inferencing, which may be performed anywhere from the cloud down to the humblest AI-equipped sensor.
AI is a greedy consumer of two things: computational processing power and data. In the case of processing power, OpenAI, the creator of ChatGPT, published the report AI and Compute, showing that since 2012, the amount of compute used in large AI training runs has doubled every 3.4 months with no indication of slowing down.
With respect to memory, a large generative AI (GenAI) model like ChatGPT-4 may have more than a trillion parameters, all of which need to be easily accessible in a way that allows to handle numerous requests simultaneously. In addition, one needs to consider the vast amounts of data that need to be streamed and processed.
To read the full article, click here
Related Semiconductor IP
- nQrux® Root of Trust IP
- AXI to UCIe Bridge IP
- UCIe 2.x Controller IP
- SWI3S (SoundWire I3S Interface) Peripheral Controller Core IP
- OpenTitan-based RISC-V Secure Element
Related Articles
- SPARC: Automated Root-Cause Analysis of Pre-Silicon Power Side-Channel Leakage in the Processor Design Flow
- The Role of Interconnection in the Evolution of Advanced Packaging Technology
- Understanding the Importance of Prerequisites in the VLSI Physical Design Stage
- The Growing Imperative Of Hardware Security Assurance In IP And SoC Design
Latest Articles
- A Secure dToF LiDAR SoC with Dual-Domain Fingerprinting and Event-Driven AFE Circuit Achieving Sensor-Level Attack Resilience
- ZTA-Q: an Open-source RISC-V Platform for Accurate Quantized CNN Inference
- Automated Pre-Silicon Verification of High-Speed DDR5 and LPDDR5/6 Memory Controllers: Closed-Loop Timing, Mode Register, and PHY Synchronization in UVM
- U-Sonic: An Open-Source 8-Channel Ultrasound Transmit IP in a 130 nm RISC-V SoC
- S-ALSA: Co-Design of Adiabatic Logic-based Sensing and Balanced Bit-Cells for Secure and Energy-Efficient MRAM