RaiderChip Hardware NPU adds Falcon-3 LLM to its supported AI models

The model, launched by Abu Dhabi’s Technology Innovation Institute (TII), runs seamlessly in RaiderChip’s AI accelerator and its FPGA based demonstrator.

Spain -- December 20, 2024 -- On December 17th, the Technology Innovation Institute (TII) of Abu Dhabi (United Arab Emirates) launched Falcon 3, a new open-source generative AI model available in four sizes: 1B, 3B, 7B, and 10B parameters. Categorized as a small model by its size, Falcon 3 delivers outstanding performance when compared to competing LLMs in its category.

Thanks to RaiderChip technology’s ease of use, implementing support for new LLM AI models is straight-forward and transparent. Only a firmware update is needed to accelerate Falcon-3 inference fully inside its hardware NPU.

RaiderChip NPU running, fully standalone and offline, the Falcon-3 1B LLM with BF16 weights

Falcon 3’s design, optimized for lightweight infrastructure efficiency, is particularly appealing to RaiderChip as it aligns with the company’s strategy to offer generative AI solutions that are embeddable on low-cost hardware with limited resources. According to Victor Lopez, RaiderChip’s CTO, “The fact that Falcon models are available in various sizes, including quantized versions, allows us to provide our customers with fast and efficient inference across a wide range of edge and embedded devices using our state-of-the-art NPU, which provides Generative AI in fully offline and standalone devices.”

The introduction of small models, capable of delivering extraordinary reasoning, comprehension, and generation capabilities across multiple languages, enhances the scope and quality of generative AI applications based on FPGAs. These include conversational assistants, recommendation systems, and data analysis platforms, among others —an expanding list expected to grow significantly in the coming months and years.

RaiderChip’s commitment to low-cost FPGAs contributes to the diversification and democratization of access to advanced artificial intelligence. This aligns with Falcon 3’s philosophy of offering powerful and accessible models to all users. Thanks to RaiderChip’s NPU, Falcon 3 can now run locally, offline, and on-premises across a greater number of new devices, unlocking new opportunities for generative AI at the edge.

Companies interested in trying the Generative AI NPU may reach out to RaiderChip for access to our FPGA demo or a consultation on how our IP cores can accelerate their AI workloads.

More information at https://raiderchip.ai/technology/hardware-ai-accelerators