Right-Sized Inference for Physical AI Silicon
Physical AI turns perception into action in real time. Autonomous vehicles, robots, drones, and aerospace systems now infer, adapt, and decide at the edge, where milliseconds matter and cloud round-trips do not fit. Bringing these capabilities to life requires a heterogeneous compute architecture that combines CPUs, DSPs, GPUs, NPUs, and other processing resources, each selected for the role it performs best. This is the second post in a five-part series on silicon design for physical AI. The first post established why physical AI demands a new class of silicon. This post explores the NPU and the vital role it plays in accelerating AI inference within these systems.

Figure 1: Physical AI and Agriculture
Developing the right physical AI solution requires more than choosing a TOPS target for the NPU. The configurable Cadence Neo NPU architecture scales from always-on sensing to demanding multi-hundred-TOPS systems, with compute capacity, memory bandwidth, local memory size, and integration flexibility that can be tailored to create a right-sized solution. The Neo NPU architecture can also form the foundation for higher levels of integration. Neo MX brings multiple Neo NPUs together in a multicore solution, while the Cadence Neo AI Chiplet integrates Neo MX with memory controllers and standards-based chiplet control. A common software flow supports these implementations, giving teams the flexibility to choose the approach that best fits their requirements. If you architect SoCs or lead silicon R&D for autonomous systems, this is the framework that connects inference ambition to production reality.
Why Physical AI Needs Edge Inference, Not Cloud AI
Edge inference processes real-world inputs on the device, enabling physical AI systems to perceive, decide, and act without waiting for a remote data center. Local execution helps deliver the low latency, reliability, deterministic performance, and security required by applications such as robotics and autonomous vehicles, while reducing the need to transmit sensitive sensor data across a network. Supporting today’s perception networks, vision transformers, and multimodal models at the edge requires AI acceleration that balances compute performance, memory bandwidth, local memory, power, and area. This is the foundation of right-sized inference.
What Is Right-Sized Inference?
Right-sized inference begins with designing the heterogeneous compute and memory architecture needed to solve a particular physical AI challenge. Architects must determine how CPUs, DSPs, GPUs, NPUs, and other processing resources will work together, as well as the memory capacity and bandwidth needed to support the targeted AI workloads.
When the discussion turns specifically to AI acceleration, performance is usually the first consideration. For physical AI systems, however, that performance must be delivered within tight power and area constraints. When architects need high inference performance while minimizing power consumption and silicon area, they turn to an NPU. Choosing an NPU is only the starting point. The next challenge is determining the right balance of compute, local memory, memory bandwidth, and other configurable resources. So, what does it mean to right-size an NPU for inference?
What Is a Right-Sized NPU for Inference?
A right-sized NPU provides the compute, local memory, and memory bandwidth needed to achieve the required AI inference performance and accuracy while meeting the power, performance, and area targets of the final product. An always-on keyword detector may be constrained to microwatts of power, while a high-end perception system may require hundreds of TOPS of AI acceleration. A fixed NPU configuration can result in underutilized silicon and inefficient power consumption at one end of that spectrum or fail to deliver the required capability at the other.
Right-sizing therefore starts with configuring the NPU for the targeted AI workloads. Architects must balance MAC count and supported data types with local memory, external memory bandwidth, area, and energy efficiency. The objective is not to maximize any single resource, but to create the combination that best meets the needs of the product.
The appropriate implementation path is another part of that decision. Cadence provides three levels of integration, allowing architects to choose the balance of customization, implementation control, and deployment simplicity that best meets their needs.
The configurable Cadence Neo NPU IP gives SoC designers the greatest control over the acceleration architecture. When higher levels of performance and integration are required, the Neo MX subsystem combines multiple Neo NPUs, delivering additional processing power, interconnect, and control resources needed for scalable AI acceleration. For systems in which a chiplet-based accelerator makes the most sense, the Cadence Neo AI Chiplet extends that same NPU architecture into a more highly integrated, standards-based solution. Together, these options allow organizations to select both the NPU configuration and the implementation path that best fits their product requirements, design capabilities, and time-to-market goals.

Figure 2: Right-size inference for physical AI
The Cadence Neo NPU: Scalable, Power-Efficient Edge Acceleration
The Cadence Neo NPU is the foundation of the AI inference acceleration solution. It is an RTL-configurable, power-efficient neural processing unit built for edge AI inference. It provides high-performance, area-optimized acceleration that scales on a common architecture from always-on IoT devices to high-throughput, transformer-class applications.
How Does the Application Drive the Right NPU Configuration?
Physical AI applications span a wide range of performance requirements. At one end, there are small, always-on functions such as keyword spotting, anomaly detection, and sensor monitoring. More demanding applications use AI for image enhancement, object detection, video analytics, radar processing, and other perception tasks. Requirements increase further with transformer-based language and vision models, robotics vision-language-action models, and systems that must run multiple networks and sensor modalities concurrently. Each application places different demands on inference performance, local memory, memory bandwidth, power, and area. Smaller, always-on applications may prioritize minimum power and area, while increasingly complex perception, transformer, and multimodal systems require greater compute capacity and more data movement. Because the application ultimately determines the resources required of the NPU, the configurable Cadence Neo NPU architecture allows designers to tailor these resources to create the right fit for each product.
How Do Architects Determine the Right NPU Configuration?
Architects often start with an AI network they need to accelerate, which might range from speech and audio processing to image segmentation and language models. They typically have a set of system-level goals, such as frame rate, power, or silicon area. Cadence software tools allow architects to compile and simulate the network against different NPU configurations, varying parameters such as MAC count, memory bandwidth, and local memory size. By iterating on these parameters, architects can understand the trade-offs and identify an NPU configuration that meets their performance and system requirements.
Cadence Neo IP Core Dual-Engine Architecture
The Neo NPU IP core uses a dual-engine design that runs in parallel:
- 3D Matrix Engine handles convolution, matrix multiply, and tensor operations, the heavy compute at the core of most neural networks.
- 2D Planar Engine handles vector and element-wise functions such as activation, normalization, and pooling.
Executing both engines concurrently keeps utilization high and avoids stalls, which translates directly into better throughput per TOPS and lower energy per inference.

Figure 3: Cadence Neo NPU IP core
What Options Are Available to Right-Size the NPU?
Compute capacity is the first of these configurable resources. The Neo NPU’s 3D Matrix Engine can be configured, allowing compute resources to be tuned to the needs of the application. The 2D Planar Engine scales with the number of MACs selected during IP configuration, providing the “Right-size” of vector and scalar processing. The 8-bit MACs also support 16-bit and 4-bit operations and can be augmented at IP generation time to also support FP16 or BF16 data types.
But inference at the edge is about more than just the right number of MAC engines. Many times, memory bandwidth and local memory capacity are equally important to overall performance and efficiency. For that, the Neo NPU provides a variety of interface widths and on-chip memory so that performance and power can be dialed in with respect to area and bandwidth. The Neo NPU allows architects to configure these three resources together, balancing network performance and power against the area and bandwidth constraints of the final product:
- MAC options: 256, 512, 1k, 2k, 4k, 8k, 16k, or 32k
- AXI interface widths of 128, 256, or 512 bits.
- On-chip unified buffer from 16 KB to 32 MB
Neo MX Subsystem: Multi-Core Acceleration Scaling Beyond a Single NPU
Physical AI requirements are becoming increasingly demanding. More complex autonomy networks, transformer-based models, and the concurrent execution of multiple sensor streams are driving a significant increase in edge-compute requirements. At the same time, these systems remain constrained by power consumption and silicon area. They must support complex perception, reasoning, and action workloads operating simultaneously under tight latency requirements. When these demands exceed the capabilities of a single NPU, designers need a multicore architecture that can scale AI acceleration efficiently. The Neo MX subsystem addresses this need by extending the Neo architecture into a scalable, multicore AI acceleration platform.
Neo MX combines multiple Neo NPUs with integrated vector processing, a Network-on-Chip (NoC), data-movement resources, and controller logic. Together, these components provide the processing and system infrastructure needed to coordinate multiple NPU cores, manage data movement, and execute complex AI workloads efficiently. This higher level of integration simplifies the development of demanding physical AI systems.
NPUs are purpose-built for efficient AI acceleration, but not every operation in an AI network can run on an NPU core. In a standalone NPU implementation, these operations may “fall back” to a host CPU or DSP, whose general-purpose processing capabilities can execute them. As MAC arrays become larger, they account for a greater share of the total accelerator area. This makes dedicated vector processing a practical addition for executing these operations without significantly increasing the overall area. Neo MX integrates vector processing directly into the subsystem, allowing the complete AI workload to remain within the accelerator. This reduces data movement and latency, helps maintain predictable performance, and leaves the host processor available for system-level tasks.

Figure 4: Cadence Neo MX subsystem
Cadence Neo AI Chiplet: In a Production-Ready Package
The Cadence Neo AI Chiplet is a complete, standards-based chiplet that packages the Neo MX subsystem into a production-ready AI accelerator. It integrates eight 16k MAC Neo NPUs with controller logic, NoC, and vector processing to deliver approximately 320 TOPS of AI acceleration in a fully self-contained, production-ready package.
Key characteristics:
- Built on the Neo MX subsystem, it shares the same architecture and software flow as the IP and subsystem paths.
- Acceleration, scalable further in multi-chiplet configurations.
- Standards-based interoperability aligned to OCP Foundation Chiplet System Architecture (FCSA) and Universal Chiplet Interconnect Express (UCIe), delivering an interoperable chiplet infrastructure.
You can use the chiplet as a reference or work with Cadence to customize it to your exact specifications, giving you a standards-based, production-ready, route to AI acceleration without designing the accelerator from scratch.

Figure 5: Cadence Neo AI Chiplet
NeuroWeave SDK: One Compiler from Evaluation to Deployment
Hardware is only half the equation. The NeuroWeave SDK is a unified, ahead-of-time software flow that transforms AI models into optimized deployments for embedded systems. One compiler serves all three integration paths, so your software investment carries across IP, subsystem, and chiplet.
Broad Model Support
Built on a modern MLIR-based compiler infrastructure, NeuroWeave imports models from PyTorch, TensorFlow, TensorFlow Lite, and ONNX. The MLIR foundation is designed to evolve with a fast-moving AI landscape.
System-Level Workload Partitioning
NeuroWeave automatically and intelligently partitions execution layers across NPUs, DSPs, and host CPUs. This maximizes overall system efficiency rather than optimizing one block in isolation.
Quantization
The SDK supports a wide range of quantization strategies, including pre-quantized models, giving you the flexibility to balance accuracy, performance, memory footprint, and power for each application.
Deployment Analysis Before Tape-Out
Understanding whether an AI architecture will meet performance, memory, bandwidth, and latency requirements before committing to silicon is a critical part of building a successful, right-sized, physical AI system. The NeuroWeave SDK enables architects to compile and evaluate AI networks early in the design process, providing insight into metrics such as frames per second (FPS), tokens per second (TPS), memory bandwidth utilization, memory footprint, and relative execution cycles across the target system.
To support various stages of development, Cadence provides multiple simulation paths ranging from fast SystemC-based exploration through cycle-accurate RTL analysis. This allows architects to balance rapid architectural exploration with increasing levels of implementation fidelity as the design matures. By evaluating real AI workloads before tape-out, teams can validate architectural decisions, identify bottlenecks early, and reduce the risk of costly late-stage design changes.

Figure 6: Unified software flow via the Cadence NeuroWeave SDK
Three Paths to Integration
The strength of the right-sized approach is choice. You deploy AI your way, on a common architecture and a single compiler:
- Neo NPU IP for custom SoC integration, sized precisely to your workload.
- Neo MX subsystem for multi-core acceleration with zero host fallback.
- Cadence Neo AI Chiplet for production-ready integration into a system-in-package.
Because all three share the Neo architecture and the NeuroWeave flow, moving between them does not force a redesign of your software or a change of toolchain. That is how you scale right, scale beyond, and scale your way.
Vertical Applications for Autonomous Systems
Right-sized inference maps directly to the demands of each autonomous market:
- Automotive autonomy: Scale vehicle autonomy on a common AI architecture, from in-cabin sensing to advanced perception, across a full vehicle platform.
- Robotics: Transform perception into intelligent action across robotics platforms, matching compute to each robot class.
- Drones: Maximize airborne intelligence within strict size, weight, and power (SWaP) constraints, where every milliwatt counts.
- Aerospace and defense: Create mission-ready AI platforms built to evolve, with acceleration that scales as requirements grow.
Key Takeaways
- Physical AI requires local, deterministic edge inference, not cloud AI.
- Right-sized inference matches AI acceleration to each workload, protecting PPA at every scale.
- The Neo NPU scales from 256 to 32,768 MACs across four tiers, from always-on to high-throughput multimodal.
- Neo MX adds multicore acceleration with zero host fallback.
- The Neo AI Chiplet delivers up to 320 TOPS in a standards-based, production-ready package.
- NeuroWeave SDK provides one MLIR-based compiler for all three integration paths.
Frequently Asked Questions
What is right-sized inference for physical AI?
Right-sized inference is AI acceleration configured to match the exact compute, memory, and bandwidth needs of a workload, rather than over-provisioning a general-purpose engine. It lets teams scale from always-on, low-power inference to complex multimodal systems while holding power, performance, and area (PPA) targets.
How does the Cadence Neo NPU scale across workloads?
The Neo NPU scales through configurable compute, memory, and bandwidth resources. MAC counts can be configured from 256 to 32,768 8-bit MACs, while interface bandwidth and on-chip memory can be tailored to the needs of the application. This flexibility allows a common Neo architecture to support everything from always-on sensing to demanding multimodal physical AI systems, all while maintaining a common architecture and software flow.
What are the product options for the Neo NPU IP?
| Product | MACs |
| Neo 210 | 256 or 512 |
| Neo 220 | 1k or 2k |
| Neo 230 | 4k or 8k |
| Neo 240 | 16k or 32k |
What is zero host fallback in the Neo MX subsystem?
Zero host fallback means the AI subsystem executes the full workload without spilling operations back to the host CPU. This avoids latency spikes and power penalties, keeps performance predictable, and frees the host CPU for other system tasks.
How much AI acceleration does the Cadence Neo AI Chiplet deliver?
The Cadence Neo AI Chiplet delivers up to 320 TOPS. It is built on the Neo MX subsystem, integrating eight 16k MAC Neo NPUs with controller logic, NoC, and vector processing, and it aligns with OCP FCSA and UCIe standards.

Figure 7: Physical AI and Automation
Moving From Inference Ambition to Shipping Silicon
Right-sized inference gives you a way to match AI acceleration to every product in your line, from a low-power sensor node to a hundreds-of-TOPS perception platform, all on one architecture and one compiler. That common foundation is what lets architects commit early, validate before tape-out, and scale across generations without restarting the software effort.
The next post in this series examines how to protect that intelligence across its full lifecycle, with trusted execution from boot through end-of-life. To put right-sized inference to work in your own design, connect with Cadence for a Neo NPU and NeuroWeave SDK evaluation.
Explore Cadence IP:
Related Semiconductor IP
- Highly scalable performance for classic and generative on-device and edge AI solutions
- NPU
- RISC-V-Based, Open Source AI Accelerator for the Edge
- NPU IP
- Highly scalable inference NPU IP for next-gen AI applications
Related Blogs
- Rivian’s autonomy breakthrough built with Arm: the compute foundation for the rise of physical AI
- World First: Synopsys MACsec IP Receives ISO/PAS 8800 Certification for Automotive and Physical AI Security
- Announcing Arm AGI CPU: The silicon foundation for the agentic AI cloud era
- How data movement defines performance for AI silicon