Vision-Language Models (VLM) – the next big thing in AI?

By Semidynamics

May 27, 2025

AI has changed a lot in the last ten years. In 2012, convolutional neural networks (CNNs) were the state of the art for computer vision. Then around 2020 vison transformers (ViTs) redefined machine learning. Now, Vision-Language Models (VLMs) are changing the game again—blending image and text understanding to power everything from autonomous vehicles to robotics to AI-driven assistants. You’ve probably heard of the biggest ones, like CLIP and DALL-E, even if you don’t know the term VLM.

Here’s the problem: most AI hardware isn’t built for this shift. The bulk of what is shipping in applications like ADAS is still focused on CNN never mind transformers. VLM? Nope.

Fixed-function Neural Processing Units (NPUs), designed for yesterday’s vison models, can’t efficiently handle VLMs’ mix of scalar, vector, and tensor operations. These models need more than just brute-force matrix math. They require:

• Efficient memory access – AI performance often bottlenecks at data movement, not computation.
• Programmable compute – Transformers rely on attention mechanisms, softmax etc. that traditional NPUs struggle with.
• Scalability – AI models evolve too fast for rigid architectures to keep up.

AI needs to be freely programable. Semidynamics provides a transparent, programable solution based on the RISC-V ISA with all the flexibility that provides.

Instead of forcing AI into one-size-fits-all accelerators, you need architectures that let you build processors better suited to your AI workload. Semidynamics’ All-In-One approach delivers all the tensor, vector and CPU functionality required in a flexible and configurable solution. Instead of locking into fixed designs, a fully configurable RISC-V processor from Semidynamics can evolve with AI models—making it ideal for workloads that demand compute designed for AI, not the other way around.

VLMs aren’t just about crunching numbers. They require a mix of vector, scalar, and matrix processing. Semidynamics’ RISC-V-based All in one compute element can:

• Process transformers efficiently—handling matrix operations and nonlinear attention mechanisms.
• Execute complex AI logic efficiently—without unnecessary compute overhead.
• Scale with new AI models—adapting as workloads evolve.

Instead of being limited by what a classic NPU can do, our processors are built for the job. Crucially they are fixing AI’s biggest bottleneck: memory bandwidth. Ask anyone working in AI acceleration—memory is the real problem, not raw compute power. If your processor spends more time waiting for data than processing it, you’re losing efficiency.

That’s why Semidynamics’ Gazzillion™ memory subsystem is a game-changer:

• Reduces memory bottlenecks – Feeds data-hungry AI models with high efficiency.
• Smarter memory access – copes with slow, external DRAM by hiding its latency.
• Dynamic prefetching – Minimizes stalls in large-scale AI inference.

For AI workloads, data movement efficiency can be as important as FLOPS. If your hardware isn’t optimized for both, you’re leaving performance on the table.

AI shouldn’t be held back by hardware limitations. That’s why RISC-V processors like our All-In-One designs are the future. And yet most RISC-V IP vendors are struggling to deliver the comprehensive range of IP needed to build VLM capable NPUs. Semidynamics is the only provider of fully configurable RISC-V IP with advanced vector processing and memory bandwidth optimization—giving AI companies the power to build hardware that keeps up with AI’s evolution.

If your AI models are evolving, why is your processor staying the same? The AI race won’t be won by companies using generic processors. Custom compute is the edge AI companies need.

Want to build an AI processor that’s made for the future? Get in touch with Semidynamics today.