RaiderChip brings Meta Llama 3.2 LLM HW acceleration to low cost FPGAs
The company incorporates the latest model, presented by Meta less than a week ago, into the catalog of LLMs already accelerated on a wide range of FPGAs
Spain -- October 1st, 2024 -- Just 6 days after its launch by Meta, RaiderChip has added support for the new Llama 3.2 model to the list of LLMs hardware accelerated by its GenAI v1 IP core for FPGAs. The supported models, which already included previous models from the same company (such as Llama 2, Llama 3, and Llama 3.1), as well as models from other providers (Microsoft’s Phi-2 and Phi-3), have been expanded again, following RaiderChip’s strategy of adding the most relevant models as they hit the market.
RaiderChip’s Generative AI hardware acceleration IP core is designed to accelerate any model built on Transformers technology, which underpins the vast majority of LLMs. The selection of additional supported models on FPGA is determined by customer choices of specific target devices and foundational LLMs. Supporting a foundational model enables any customer-specific fine-tuned derivative to be accelerated seamlessly, without the need to share its weights.
It is important to note that the various FPGA options available on the market have different sizes and variable logic and memory capacities. These technical factors, combined with commercial decisions such as unit cost, power consumption, or final functionality, mean that the ideal FPGA and LLM vary for each potential final product. For instance, using “smaller” models (such as Meta’s Llama 3.2 1B and Microsoft’s Phi-2 2.7B) or 4-bits Quantization are ideal for products based on simpler, more economical FPGAs; while larger LLMs in their original format, preserving the model’s full floating point precision, require larger and more expensive FPGAs,” explains Victor Lopez, the company’s CTO.
GenAI v1-Q running the Llama 3.2 1B LLM model with 4 bits Quantization on a low-cost Versal FPGA with LPDDR4 memory
Like previous models, the new Llama 3.2 can be tested through an interactive demo based on a Versal FPGA. “The demonstrator exposes not only a simple local and remote API access to the accelerated LLM, but may also be run directly through a chat terminal application that allows users to interact with the different models, while they are HW accelerated by RaiderChip’s IP core on the FPGA. At RaiderChip, we invite our customers to personally experience the real performance of our product, assessing key aspects like intelligence, latency, or tokens per second in a live demonstrator, beyond the usual performance tables based on theoretical or simulation data, which are often confusing,” the team comments.
Companies interested in trying the GenAI v1-Q IP core may reach out to RaiderChip for access to our FPGA demo or a consultation on how our IP cores can accelerate their AI workloads.
More information at https://raiderchip.ai/technology/hardware-ai-accelerators
Related Semiconductor IP
- MACsec Engine optimized for 10Base-T1S/T1L applications
- Dual Port, Voltage Scaling Register File Compiler on GF22FDX+
- Single Port, Voltage Scaling, High Speed SRAM Memory Compiler on GF22FDX+
- I3C Full Controller / Target
- I3C HCI Host Controller
Related News
- Movidius Beefs up HW Acceleration in AI Chip
- NSCore, Inc. Introduces its Automotive Grade-1 Qualified Non-Volatile Memory solution to help address the increasing need for Low Cost Semiconductor Chips
- Efinix Low Power, Small Footprint FPGA Selected for SPRESENSE Development Platform
- Lattice Extends Low Power Leadership with New Lattice Avant FPGA Platform
Latest News
- Cyient Semiconductors Secures Strategic Investment from GlobalFoundries
- JEDEC® Previews Next-Generation DDR6 DRAM: Higher Performance, Greater Capacity, Improved Power Efficiency and Data-Center-Class Reliability
- INTELPRO Adopts Andes AX46MPV Flagship Multicore Vector Processor and AndesAIRE™ AnDLA™ I370 to Power Next-Generation Heterogeneous AI and Robotics Solutions
- VeriSilicon Introduces Low-Power, High-Performance Wearable Subsystem Solution for Next-Generation AI/AR Glasses
- IC Manage Announces MCP Server for GDP-AI Design and IP Management System
