The realities of developing embedded neural networks

By Tony King-Smith, AImotive

Much has been written about using AI for increasingly smart vehicles. But how do you take a neural network (NN) developed on a server farm and squeeze it into resource-constrained embedded hardware in production cars? This article explores how we should empower automotive production AI R&D engineers to refine NNs throughout the process of taking NNs from prototype to production, rather than today’s process of handing over an NN to an embedded software team too early.

Embedded AI: Embedded software – but not as we know it

With any embedded software destined for deployment in volume production, an enormous amount of effort goes into the code once the implementation of its core functionality has been completed and verified. This optimization phase is all about minimizing memory, CPU and other resources needed so that as much as possible of the software functionality is preserved, while the resources needed to execute it are reduced to the absolute minimum possible.

This process of creating embedded software from lab-based algorithms enables production engineers to cost-engineer software functionality into a mass-production ready form, requiring far cheaper, less capable chips and hardware than the massive compute datacenter used to develop it. However, it usually requires the functionality to be frozen from the beginning, with code modifications only done to improve the way the algorithms themselves are executed. For most software, that is fine: indeed, it enables a rigorous verification methodology to be used to ensure the embedding process retains all the functionality needed.

However, when embedding NN-based AI algorithms, that can be a major problem. Why? Because by freezing the functionality from the beginning, you are removing one of the main ways in which the execution can be optimized.

What is the problem?

Click here to read more ...