Designing for What Comes Next in AI

By Yaron Galitzky

October 29, 2026

AI is expanding into everything, from always-on sensors and cameras to vehicles, robotics, personal computing and many other devices. At the same time, workloads are rapidly expanding beyond CNNs to include transformers, multimodal models, generative AI and agentic AI workloads.

For years, selecting an AI processor was largely about matching performance to a product with a defined use case: compute, power, silicon area and the models it needed to run. Those considerations still matter, but they are no longer enough. Semiconductor and device companies increasingly need to support multiple product categories and rapidly evolving workloads, each requiring a different balance of compute, memory, bandwidth, power and cost.

Choosing an NPU is therefore no longer simply a decision about the right compute engine for one design. It is a decision about the architectural and software foundation for multiple product lines for several generations.

Working closely with more than 15 customers who licensed the first generation of Ceva-NeuPro NPUs reinforced this observation: successful AI products are not built simply by maximizing TOPS. They are built by finding the right balance of compute, memory, bandwidth, power and cost for each use case, combined with the flexibility to support new workloads as they emerge.

Today, we are introducing the next generation of the Ceva-NeuPro NPU portfolio, built on those lessons and designed for what comes next.

What We Learned from Our Customers

We saw the practical impact of that lesson across the wide range of products our customers are building: there is no one-size-fits-all approach to AI processing.

An always-on sensor operates within an extremely constrained power and area budget. A smart camera needs high-resolution processing at low latency. Robotics may combine perception with multimodal and agentic AI, while AI infrastructure must support complex models together with significant memory and data-movement demands.

What has stood out to me across our customer engagements is that the best technical answer is rarely the most powerful engine. It is the one that best fits the workload today while giving customers room to grow tomorrow.

This becomes particularly important with Physical AI, where intelligence is increasingly distributed across sensing, perception and reasoning, with different workloads running where they can be executed most effectively.

The right AI architecture is not defined by compute alone. It delivers the right intelligence, in the right place, within the system constraints.

Customers need the flexibility to right-size AI processing throughout the system without starting over at each performance tier.

Designing for the Next Generation of AI

The future of AI will be determined not only by how much intelligence we can create, but by how efficiently we can deploy it.

This is something I care deeply about. As AI expands into billions of devices, our challenge is not simply to deliver more compute, but to deliver more useful intelligence with less energy and fewer system resources. Efficiency must be designed into the architecture from the start, not added as an afterthought.

That belief helped shape the next-generation NeuPro architecture. As AI moves toward increasingly sophisticated transformer, multimodal, generative and agentic workloads, architectures need to deliver more than peak compute. They need to run the most advanced models efficiently, balancing compute, memory, bandwidth, data movement and power.

The next-generation NeuPro architecture was designed with this in mind, delivering up to 2X higher AI throughput per mm² and a 50% improvement in energy efficiency compared with the first generation.

Delivering useful AI performance requires improving more than the compute engine itself. FP4 and FP8 mixed precision reduce the memory and data movement required for inference. Higher memory bandwidth, dynamic shapes and paged KV cache address increasingly memory-intensive transformer workloads and longer contexts.

Higher MAC utilization through hardware tensor reshaping helps turn peak compute into effective inference performance. The NeuPro architecture reshapes tensor dimensions in hardware, including space-to-depth and tile-to-depth transformations, helping keep the MAC array efficiently utilized across diverse layer shapes, from shallow early-stage layers to transformer token grids. This helps translate the available peak compute into effective performance across a wider range of real-world workloads.

The architecture also supports emerging model structures, including Mixture-of-Experts and linear attention. Pre-quantized models and advanced compression, quantization and pruning help customers deploy and optimize models more efficiently.

We also applied lessons from first-generation NeuPro customer implementations to the physical design of the next generation. The architecture is designed with the physical realities of silicon in mind, optimizing compute, memory and data movement so that efficiency gains translate into real implementations. Built-in peak power control manages current surges as compute resources start and stop, supporting higher operating frequencies and multi-engine scaling without sacrificing throughput.

Taken together, these capabilities address the compute, memory, data movement and power challenges that increasingly determine real-world AI performance, helping translate peak compute into useful system-level performance.

But designing for the next generation means more than supporting today’s most advanced models. It means creating an architecture that can scale across very different workloads and remain flexible as AI evolves.

The constraints are different at every point in the system. At the edge, every milliwatt can affect battery life and thermal design, while every square millimeter can affect product cost and viability. In AI infrastructure, memory bandwidth and data movement between memory, compute and networking subsystems can become central constraints.

Ultimately, the objective is the same: optimize the efficiency of the entire system to deliver more useful intelligence within its power, area, memory and bandwidth constraints.

One Scalable NPU Family, One Architecture, One SDK

The next-generation NeuPro portfolio translates these principles into four workload-optimized product lines, each designed for a different balance of AI performance and system constraints:

• NeuPro Nano for always-on sensing and entry-level perception in power-, area- and cost-constrained devices
• NeuPro Prime for mainstream CNN and transformer workloads
• NeuPro Max for advanced edge AI, including high-resolution perception, multimodal models, on-device assistants, robotics, Physical AI and agentic AI
• NeuPro Ultra for AI infrastructure, including complex large language models in cloud and data-center inference and neural processing within memory and connectivity subsystems.

The key is not simply having four NPUs. It is that they are built on a common architecture and supported by a unified software environment. Customers can select the processing capabilities that best match each workload without adopting a different NPU architecture and development environment for every product.

This also creates flexibility at the system level. Multiple NeuPro engines can be combined within a single SoC, allowing different workloads to run on appropriately sized processing resources rather than overprovisioning one engine for every task.

The principle is simple: right-size the AI processing for each workload while keeping the architecture, tools and software investment consistent as products scale.

Software Is as Strategic as Hardware

When introducing a new NPU generation, hardware performance naturally gets much of the attention. But software determines how effectively customers can carry that investment across products and as models evolve.

An NPU core is integrated into a product once, but the software environment must support that product throughout its lifecycle. Developers need to evaluate new models, optimize and quantize them, compile applications, analyze performance and deploy inference as models and frameworks continue to change.

This is why a common architecture alone is not enough. If every performance tier requires a different toolchain or development methodology, customers lose much of the benefit of having a scalable NPU family.

The next-generation NeuPro portfolio is therefore supported by a unified software development environment across the family. Customers can use common model-evaluation, optimization, compilation and runtime flows as they scale from constrained devices to advanced edge AI and multicore infrastructure systems.

Our goal is simple: as customers scale across the NeuPro portfolio, their software investment should scale with them, preserving their tools, workflows and engineering expertise.

Designing for Models We Haven’t Seen Yet

AI is evolving faster than any single product cycle. We designed the next generation of NeuPro to efficiently run today’s most advanced models, with the flexibility to support what comes next.

We cannot predict exactly which model architectures, numerical formats or workloads will define the coming years. What we can predict is continued change.

Future readiness therefore cannot come from optimizing for one model or one moment in the market. It has to be built into the architecture and software from the beginning.

Supporting new operators is only part of the challenge. New models can also introduce new system-level requirements, such as KV-cache management and Mixture-of-Experts routing. NeuPro is designed to address both: new operators can run on a vector processing unit (VPU) integrated into the hardware pipeline, while the flexible pipeline architecture enables new system flows.

The goal is to support the most advanced AI models of today while giving customers a platform that can evolve with the models, workloads and products of tomorrow.

Explore Ceva IP:

• NPU IP
• NPU IP
• NPU IP
• NPU IP for Embedded ML

Why the Right IP Partner Matters

Choosing an NPU is no longer just about licensing an IP block. It is a long-term decision about the technology and expertise that will support an AI product roadmap as models, workloads and system requirements continue to evolve.

The right AI IP partner therefore needs to bring more than RTL. It needs scalable architecture, a unified software environment and experience gained from working with customers on real AI products. Just as importantly, it needs to understand that different workloads require different solutions.

For me, this is one of the most important lessons from working with first-generation NeuPro customers: the value of an AI platform is measured not only by what it enables today, but by how well it prepares customers for what comes next.

That is what we set out to create with the next generation of NeuPro: not simply a family of NPUs, but a scalable AI foundation designed to help our customers build what comes next.