Protecting Edge AI with RISC-V CHERI: See how easily memory-safe security can be added to an AI application (part 1)

As edge AI expands into industrial and infrastructure applications, securing the entire software stack becomes critical. We will show that running edge AI workloads with CHERI memory protection is possible today, without custom silicon or extensive software redevelopment. Codasip has demonstrated this by successfully running an AI inference application on the Codasip Prime platform, a RISC-V CHERI-enabled Linux field-programmable gate array (FPGA) platform, paired with a Hailo-10H edge AI accelerator.

The software adaptation took just one senior engineer's afternoon and involved only around 100 lines of code changed or added across Hailo's PCI Express (PCIe) driver, runtime libraries and application software, with no modifications to the accelerator firmware. This demonstrates that a commercial AI accelerator can be integrated into a CHERI pure-capability environment with minimal porting effort, providing a practical foundation for more secure edge AI workloads.

We will walk you through the hardware integration, software adaptations and successful execution of the AI inference application. In Part 2, we'll explore how CHERI compartmentalisation can provide stronger isolation between the components of an AI workload.

Why combine AI acceleration with CHERI?

Edge AI is increasingly used in systems that process sensitive information and interact with the physical world. Smart energy management, industrial automation, robotics and connected infrastructure are all examples. These applications need more than inference performance. The software responsible for collecting data, communicating with accelerators and handling predictions also matters.

CHERI (Capability Hardware Enhanced RISC Instructions) extends the RISC-V processor architecture with a new hardware-enforced pointer type called a capability. Unlike a conventional pointer, which primarily holds a memory address, a capability also carries information defining the memory region it can access, the operations permitted (such as reading, writing or executing), and a hardware-protected validity tag. The processor checks these properties whenever a capability is used, preventing operations such as accessing memory outside its permitted bounds or forging pointers to gain unauthorised access.

CHERI supports two software models. In hybrid (mixed-capability) mode, conventional pointers coexist with CHERI capabilities, allowing developers to introduce capability protection selectively while maintaining compatibility with existing software. In pure-capability (purecap) mode, all language-level pointers are represented as capabilities, extending hardware-enforced memory protection throughout the compiled application and its supporting libraries. For our demonstration, we chose purecap to run the AI application and its supporting software stack with CHERI's memory-safety protections. This also provides a foundation for introducing finer-grained software compartmentalisation, which we'll explore in Part 2.

That makes CHERI particularly interesting as a host for AI accelerators. The accelerator can concentrate on executing neural networks, while the CHERI-enabled CPU runs the surrounding application and runtime software with stronger memory-safety guarantees.

But can we connect the two using existing hardware and software? That's what we wanted to demonstrate.

The hardware: Codasip Prime meets Hailo-10H

Our setup starts with Codasip Prime, Codasip's FPGA-based CHERI-RISC-V exploration platform. For this demonstration, we're using a quadcore cluster of Codasip X730 64-bit application processors, running CHERI-enabled Linux. Prime already provides the hardware and software environment needed to develop and run CHERI applications, including a C/C++ toolchain and Yocto-based Linux build environment.

For AI acceleration, we selected the Hailo-10H, a commercially available edge AI accelerator designed for workloads including neural network inference and generative AI. The Hailo-10H is available in an M.2 module form factor and communicates over PCIe. Hailo specifies up to 40 tera-operations per second (TOPS) for 4-bit integer workloads, or 20 TOPS for 8-bit integer workloads, depending on the configuration. Importantly, it also provides a Linux driver and a C/C++ runtime interface.

The Codasip X730 CPUs run Linux, manage the application, prepare data and communicate with the accelerator. The Hailo-10H executes the compiled neural network.

The kit:

Part	What it is	Key details
Codasip Prime	FPGA exploration platform for CHERI	X730 64-bit RISC-V CHERI application CPU, here as a 4-core cluster; CHERI Linux; CHERI C/C++ toolchain; Yocto build
Hailo-10H M.2	M.2 Key M AI accelerator	40 TOPS INT4 / 20 TOPS INT8; PCIe Gen 3 x4; 4 or 8 GB LPDDR4/4X on module; 2.5 W typical
Workstation x86 Linux - Ubuntu works best, Hailo also provide a docker image	Model compiler host	Runs Hailo's Dataflow Compiler in Docker; never touches the board at runtime

Step 1: Bringing Hailo's software into our Yocto environment

Hailo provides a Yocto layer, meta-hailo, containing recipes for integrating its software into embedded Linux environments. We incorporated the relevant Hailo recipes into our Codasip Prime Yocto project, then adapted the source code and dependencies for our CHERI-RISC-V target.

The main components were the Linux PCIe driver and Hailo’s library - libhailort. We are happy to provide the Yocto recipies we used to any interested prime users.

Porting the PCIe driver

The PCIe driver manages communication between the host processor and the accelerator, including device initialisation, firmware loading and data transfer. Because it had relatively few external dependencies, the driver was straightforward to port. Most changes were the familiar adjustments encountered when moving existing C code to a capability-aware architecture: clarifying pointer types, reviewing integer-to-pointer conversions and making sure memory layouts and alignment assumptions were valid.

A typical “CHERI” edit to a driver might just be something like this:

-static long hailo_query_driver_info(struct hailo_pcie_board *board, unsigned long arg)
+static long hailo_query_driver_info(struct hailo_pcie_board *board, void __user * arg)

The Hailo firmware itself did not need to be modified. We used the existing firmware supplied by Hailo, loaded onto the accelerator through the driver running on the Codasip system. We were adapting the software running on the CHERI host, not rewriting the accelerator's internal software.

Porting HailoRT

The next component was HailoRT, Hailo's runtime software for interacting with the accelerator. HailoRT provides the host-side functionality needed to load compiled models, configure inference and transfer input and output data. Its dependencies made this part slightly more involved.

In particular, we needed CHERI-compatible versions of Google's Protocol Buffers (protobuf) and the spdlog C++ logging library. Fortunately, the University of Cambridge CHERI work had already produced ports of these libraries. Rather than starting from scratch, we updated the relevant source references in our Yocto recipes to use the CHERI-adapted versions.

etchContent_Declare(
     protobuf
-    GIT_REPOSITORY  https://github.com/protocolbuffers/protobuf.git
-    GIT_TAG         f0dc78d7e6e331b8c6bb2d5283e06aa26883ca7c # v21.12
+    GIT_REPOSITORY  https://github.com/CTSRD-CHERI/protobuf.git
+    GIT_TAG         48004779f8b52659efbe8519a8bccda7dbf664b6

We then worked through the remaining HailoRT changes. As with the driver, many fixes involved assumptions that conventional C/C++ software can make about pointers but that are not valid in a purecap environment.

For example, code might assume that pointers can be freely converted to integers and back, or that a packed structure can contain pointer fields without special alignment considerations. These assumptions need reviewing when pointers carry capability metadata and must satisfy architectural validity and alignment rules. The changes were manageable, and the result was a working Hailo runtime on the Codasip Prime CHERI software stack.

This is one of the useful lessons from the exercise: porting an existing AI software stack to CHERI does not necessarily require rewriting the application or its supporting libraries. Much of the work consists of identifying and correcting existing assumptions about memory and pointers. In fact there aren't many changes at all, most of the diff is editing the build system and Yocto layer to pull in the CHERI versions of dependencies.

git diff --stat f51959034a8a49b7ee53cd0034a161faa30352ba
 hailort/cmake/external/protobuf.cmake                  | 69 +++++----------------------------------------------------------
 hailort/common/genai/serializer/CMakeLists.txt         |  2 +-
 hailort/common/os/posix/filesystem.cpp                 |  1 +
 hailort/common/utils.hpp                               |  2 +-
 hailort/drivers/common/hailo_ioctl_common.h            |  4 ++--
 hailort/hrpc_protocol/CMakeLists.txt                   |  4 ++--
 hailort/libhailort/CMakeLists.txt                      |  6 +++---
 hailort/libhailort/src/hef/hef.cpp                     |  6 +++---
 hailort/libhailort/src/vdma/driver/hailort_driver.cpp  |  0
 hailort/libhailort/src/vdma/driver/hailort_driver.hpp  |  0
 hailort/libhailort/src/vdma/memory/descriptor_list.cpp |  3 ++-
 hailort/libhailort/src/vdma/memory/mapped_buffer.cpp   |  2 +-
 hailort/libhailort/src/vdma/transfer_common.hpp        |  4 ++--
 hailort/libhailort/src/vdma/vdma_device.cpp            |  6 +++---
 hailort/libhailort/src/vdma/vdma_stream.cpp            |  3 ++-
 15 files changed, 28 insertions(+), 84 deletions(-)

Step 2: Connecting the accelerator

The Hailo-10H M.2 module was connected to the Codasip Prime system through its PCIe interface, using suitable M.2 connectivity. No custom AI accelerator silicon or redesign of the soft processor was required.

Of course, connecting a PCIe device is only the beginning. The Linux kernel still needs to enumerate the device, initialise its driver and establish reliable communication.

In our case, some configuration was needed for compatibility with the Xilinx PCIe controller used in the FPGA platform. The following options are provided to the kernel module:

/etc/modprobe.d/hailo1x.conf:
options hailo1x-pci async_probe no_power_mode=1

no_power_mode=1 prevents PCIe power-state transitions that the host configuration does not support. async_probe prevents some busy waiting that can be a little slow.

Once the kernel module is loaded, it should be visible under sysfs

root@codasip-prime:~# lspci
...
0001:01:00.0 Co-processor: Hailo Technologies Ltd. Device 45c4 (rev 01)

This confirmed that the CHERI-enabled Linux system could discover and communicate with the attached hardware.

Codasip Prime - A Xilinx VCU118-based CHERI prototyping platform.

The Hailo card (highlighted) is mounted to one of the two PCIe slots of the FPGA Drive daughter card.

Step 3: Building a model for the Hailo-10H

For our demonstration, we chose a practical edge AI workload: predicting household energy consumption. The application is relevant to homes equipped with solar panels and battery storage, where accurate energy-demand predictions could help optimise consumption and storage decisions.

The model is deliberately compact. We wanted to demonstrate the integration of AI inference into a CHERI-based system rather than build the largest possible model.

Our starting point was a model developed using PyTorch, however, a trained PyTorch model cannot simply be copied to the Hailo accelerator and executed. It first needs to be translated and compiled into a representation suitable for the accelerator.

From PyTorch to Hailo Executable Format

The first step was to export the trained floating-point model to Open Neural Network Exchange (ONNX), a widely used format for exchanging neural network models between frameworks.

Hailo provides its Dataflow Compiler to perform the conversion - its workflow includes parsing the ONNX model, validating operations against the selected accelerator architecture, optimising and quantising the network, and compiling the result into a Hailo Executable Format (HEF) file.

[info] Hailo HW architecture: hailo10h
[info] Compiling network
[info] To achieve optimal performance, set the compiler_optimization_level to "max" by adding performance_param(compiler_optimization_level=max) to the model script. Note that this may increase compilation time.
[info] Loading network parameters
[info] Starting Hailo allocation and compilation flow
[info] model_compute Successful Pre Builder (duration: 0s) 100%
[info] Building optimization options for network layers... 100%
Building pyramids for node: conv5                                                                               
[info] Successfully built optimization options - 1s 66ms
[info] model_compute Successful Pre Partition (duration: 1s)
[info] Trying to compile the network in a single context
[info] Using Single-context flow
[info] Resources optimization params: max_control_utilization=75%, max_compute_utilization=75%, max_compute_16bit_utilization=75%, max_memory_utilization (weights)=75%, max_input_aligner_utilization=75%, max_apu_utilization=75%
[info] model_compute Successful Single Context Partition (duration: 4s)
[info] Validating layers feasibility

Validating model_compute_context_0 layer by layer (100%)

 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +
 +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +  +

● Finished

[info] Layers feasibility validated successfully
[info] Running resources allocation (mapping) flow, time per context: 59m 59s
Context:0/0 Iteration 4: Trying parallel mapping...
          cluster_0  cluster_1  cluster_2  cluster_3  cluster_4  prepost
 worker0  V          V          X          V          V          V
 worker1  V          V          V          V          V          V
 worker2  V          V          V          V          V          V
 worker3  V          *          V          V          V          V

  00:00
Reverts on cluster mapping: 0
Reverts on inter-cluster connectivity: 0
Reverts on pre-mapping validation: 0
Reverts on split failed: 0

[info] Iterations: 4
Reverts on cluster mapping: 1
Reverts on inter-cluster connectivity: 0
Reverts on pre-mapping validation: 0
Reverts on split failed: 0
[info] model_compute Successful Allocation (duration: 0s)
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 62.5%               | 25%                 | 18.8%              |
[info] | cluster_1 | 100%                | 41.7%               | 44.5%              |
[info] | cluster_2 | 18.8%               | 8.3%                | 10.2%              |
[info] | cluster_3 | 100%                | 41.7%               | 40.6%              |
[info] | cluster_4 | 93.8%               | 35.4%               | 34.4%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 75%                 | 30.4%               | 29.7%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] model_compute Successful Partition and Allocation (duration: 6s)
[info] Compiling kernels of model_compute_context_0...
[info] model_compute Successful Kernel Compilation (duration: 0s)
[info] Building HEF...
[info] model_compute Successful Compilation (duration: 7s)
[info] Compilation complete
[info] Saved HEF to: /tmp/output/model_compute.hef
[info] Saved HAR to: /tmp/output/model_compute_compiled.har

In our case, the model was quantised to 8-bit integer (INT8) representation. Quantisation reduces the precision of numerical values used by the network, enabling efficient execution on hardware designed for lower-precision operations. The calibration data helps the compiler determine appropriate numerical ranges.

The generated .hef file contains the compiled model representation required by HailoRT.

We performed this compilation separately from Codasip Prime using Hailo's model-development tools, which are also available to install on ubuntu, or through a container-based environment. That separation is useful: model development and compilation can remain on a conventional development workstation, while the final inference application runs on the CHERI-enabled target.

Step 4: Running inference from a CHERI application

We now had all the necessary pieces.

Codasip Prime was running CHERI-enabled Linux, the Hailo PCIe driver and HailoRT had been ported, and the energy-prediction model had been compiled into a HEF file. The remaining step was to connect them through an application.

Many AI tutorials rely on Python, including tutorials for preparing and running models. For this project, however, we were targeting a CHERI purecap software environment without the required Python support. Fortunately, HailoRT also provides native C/C++ interfaces. We used these to build an application that interacts directly with the runtime. At a high level, the application discovers the Hailo device, opens the compiled model, configures the inference operation, prepares the model's input data and submits it for execution. The accelerator processes the data and returns its predictions to the host application.

The result is a conventional edge AI inference workflow, but with a significant difference: the host application and runtime execute as CHERI software on Codasip's X730 processors. We did not need a Python interpreter running on the target, nor did we need to modify the compiled accelerator model to accommodate CHERI. For developers already familiar with Hailo's C++ interfaces (or by reading this blog), this provides a practical route to experimenting with AI acceleration on Codasip Prime.

We set up the server to accept inferences over a network. We send a header with input information and then a payload containing the model input. We can now send an inference and watch the server process it. 

What have we demonstrated?

The result is straightforward: a Hailo-10H accelerator can be integrated with Codasip Prime and used to execute AI inference from a CHERI purecap host environment.

We have demonstrated the software path from a CHERI-enabled Linux application through HailoRT and the PCIe driver to the accelerator. We also established that the necessary software adaptations were practical, reusing existing CHERI dependency ports where available.

CHERI does not mean abandoning existing software frameworks or starting every project from zero. In this example, we used an existing Linux driver, an established AI runtime, open-source libraries, Yocto and familiar C/C++ application interfaces. The work involved adapting those components to the CHERI programming model, rather than replacing the AI software ecosystem.

There is also an important security distinction. CHERI's capability protections apply to memory accesses made through the CHERI-enabled processor and its software environment. They do not automatically restrict independent direct memory access (DMA) operations performed by an external PCIe device. Those require suitable platform-level isolation mechanisms. Likewise, compiling software in purecapmode does not automatically establish separate, least-privilege compartments for every software component.

That is where we want to go next.

Next: compartmentalising the AI workload

We now have a working foundation for more advanced experiments.

Today, the application and supporting runtime can execute in a CHERI purecap environment. The next stage is to explore more fine-grained separation between different parts of the software. For example, could model management, input preparation, inference handling and the application logic operate with independently restricted access to memory and resources? Could a component responsible for invoking AI inference be prevented from accessing unrelated application data? And in an energy-management application, could we separate the component generating predictions from trusted logic responsible for making control decisions?

These are the kinds of questions CHERI compartmentalisation is designed to help address.

In Part 2, we'll investigate how to introduce stronger software boundaries around the AI workload and discuss where this approach could lead.

Try AI acceleration on Codasip Prime

If you're already developing on Codasip Prime, the Hailo-10H provides an interesting way to extend your experiments beyond conventional CPU workloads.

With a commercially available accelerator, a CHERI-enabled Linux environment and existing C/C++ interfaces, it is possible to start exploring real AI applications on a memory-safer host. And if you're evaluating CHERI for a future edge AI product, Codasip Prime provides a platform for experimenting with these integrations before committing to silicon.

Our experience shows that combining CHERI and AI acceleration is something we can build and run today with minimal porting.

Learn more about Codasip Prime, the Hailo-10H and the CHERI C/C++ programming model. Contact Codasip for information about the development platform and the software environment used in this demonstration. Codasip Prime and Hailo 10H CHERI integration stack are available now.

Learn more about Codasip Prime