It’s often just a line item: “On-device AI,” slotted right in between connectivity features and battery performance, almost like an afterthought. But its full weight becomes immediately apparent at the design level, almost dauntingly so. The memory it consumes, the power it draws, the models that need to be built, component integration, footprint, BoM. It’s a lot, especially for ultra-low-power devices, which is why Edge AI has remained a specialist craft.
The classic route was to offload. Have inference run on a server, keep the hardware simple, job done. But we live in a different world now with mounting expectations for devices in the field. In wearables and safety-critical applications, decisions need to happen in milliseconds. In healthcare and medical applications, data privacy is a prime concern. Devices need to keep working when the connection drops. Local inference is radically more power-efficient than streaming things to the cloud. And edge workloads tend to be narrow and well-defined, which is precisely what tiny models are good at, across billions of deployed devices. Together, these factors are pushing intelligence to the edge. So, how do we get past the technical and usability hurdles to serve this need from the outset?
Where power and memory become architecture
Power is where the numbers are at their most unforgiving. Bluetooth Low-Energy (BLE), frequently the technology of choice for battery-powered devices, is designed for devices that spend most of their time asleep, with the radio being the largest single energy drain when active. The remaining headroom is tight by design. Running inference workloads on general-purpose CPUs or discrete hardware is a sure-fire way to exceed it, with knock-on effects on battery life that ultimately limit the scope of end applications a given power budget can support.
Memory access and size compound the problem. Neural network weights and activations need to live in RAM. Data movement between the CPU and the memory often chokes inference times. To be clear, this is a general architectural problem, not only a CPU-specific one. Then, for SoCs with RAM sizes in the hundreds of KB, fitting TensorFlow Lite models that do meaningful work can require significant optimization and compression, often by teams who aren’t ML specialists.
And then there’s the model development itself. This is an iterative, expertise-heavy process. It takes time to build and validate high-quality models against real-world data. With a discrete accelerator as another component to source, integrate, and justify within often very tight BoM budgets, it’s easy to see that an ultra-low-power edge AI offering needs a systemic approach to its design, built from the ground up.
Constraints become the catalyst
This is what purpose-built Bluetooth LE SoCs with integrated AI acceleration are made for. A Neural Processing Unit (NPU) on the same SoC minimizes power and integration challenges in a single move. Workloads that would have run on the CPU and drained the battery now run efficiently with minimal involvement of the main core, keeping it free for its intended tasks and at significantly lower power. Nordic’s nRF54LM20B, with its integrated Axon NPU, for example, runs the same models up to 15x faster than on the CPU alone, and at significantly lower energy consumption.
And with powerful enough software tooling, barriers to ML expertise can be minimized or even removed entirely. No-code platforms like Nordic’s Edge AI Lab can automatically generate custom models. Developers simply upload labeled sensor data (e.g., as a CSV file), configure signal processing and feature extraction settings (or even let the platform choose them automatically), and an AutoML framework called Neuton grows the network neuron by neuron, constructing the smallest accurate model possible without any manual architectural decisions.
The output is a compiled C library ready to deploy. Deployed models typically come in under 5KB, small enough to run efficiently on the CPU without needing dedicated acceleration. For more demanding ML workloads or teams that need greater control over model design, Nordic’s Edge AI Lab provides model-building and training capabilities to accelerate model training on the Axon NPU. Nevertheless, lowering the barrier to entry remains a top concern at Nordic, across AutoML and the broader model-building and deployment pipeline. The Edge AI Lab’s ability to generate wake-word detection models from typed text alone, with no audio data required, is an expression of that direction.
Customers building edge AI products today aren’t only asking for a great wireless chip. They’re asking for the full path from device to cloud and back, from model building to deployment to observability of a fielded device that can be updated over the air. This is platform infrastructure enabling the maintenance of devices on the ground for years. It refocuses engineering efforts away from complex balancing acts toward bringing exciting use cases to life. Democratizing ultra-low-power edge AI. That’s the direction Nordic is building toward.
www.nordicsemi.com