Category: Computing

The hardware that makes AI possible is hidden away in data centers, huge non-descript buildings that, because of security concerns, may not reveal their purpose. But their thermal signature is a dead giveaway. So much energy is needed to cool down the servers inside that future mega-sized data centers must be big energy production facilities, such as hydroelectric dams. That may not be enough. They are on the verge of bringing back nuclear reactors.

To counter the surge required by present and future data centers is a small but growing movement to reduce the need for computing done by data centers. By doing the computing on the edge, rather than in the (data) center, could we not stop the madness of building ever bigger data centers? OpenAI, the company that created ChatGPT, has plans to build several 5 GW data centers, 30 times the size of the biggest data centers currently in existence. 5 GW is roughly the amount of energy all of Iceland consumes in a year!

The argument for more edge processing is sound. Sensors, such as those in consumer devices and industrial equipment, emit a steady stream of data that is currently processed by hordes of GPUs on neural network chips in data center servers. These servers are always on, always demanding energy and generating heat. Yet, most of the data they are processing is just noise.

Filtering out the noise at the edge makes sense. And if the useful signal remaining could be processed on the edge, we could actually downsize our need for data centers.

That’s the theory behind a new crop of small spiking neural processors (SNPs) in a nutshell – literally. A nutshell could actually hold several SNPs.

Hey, Buddy. Want to Buy a SNP?

I was shown my first SNP in a hotel room.

I meet Innatera’s chief commercial officer, Shreyas Derashri and CEO, Sumeet Kumar, in a suite at the Venetian Hotel in Las Vegas. As companies sometimes do, Innatera has booked a room instead of a booth on the show floor. Smart. They have my undivided attention. The show floors are a circus. A suite lets the company set up rooms for different applications of their chip.

The Innatera team has traveled to Las Vegas from HQ in Delft, Netherlands, to show off the first-ever always-on spiking neural processor, the T1. Tiny beyond belief, just 6mm square, the T1 is indeed a marvel of miniaturization. Its diminutive size gives it a big advantage over NVIDIA’s processors on the big stage at CES. Yet, this little chip can do quite a bit of what its more famous counterpart can do with just 1/500 the power.

What Can It Do?

The T1’s lower-than-low power consumption opens up a flood of new product improvements. Think of portable devices, such as smartphones, wearables (like Fitbit and smartwatches), smart speakers, and smart appliances, hundreds of which are on display a few floors below, all able to last for weeks or months without being recharged or having to replace their batteries. Think also of commercial buildings and smart homes loaded with battery-operated smart and connected devices, such as doorbell cams, smoke alarms, thermostats, people counters, security systems, etc., that don’t have to be wired or have their batteries changed from the top of tall ladders.  

In all these smart devices, sensors abound. A home smart speaker may have two sensors, a commercial building, hundreds. Innatera estimates that four billion smart devices will be added each year.

I am reminded of my smart doorbell, which seems to run low on charge at the most inopportune times, such as during my vacation, and uses its precious battery to inform me of stray cats, blowing leaves, passing cars, and other things I care little about. Could it just let me know of package delivery or who is ringing the bell?

How SNPs Work

A spiking neural processor, such as the one developed by Innatera, is inspired by the functioning of biological neurons in the brain. Instead of continuously processing information like traditional neural processors, SNPs become active only when they encounter discrete, event-driven signals called “spikes,” e.g., a change in input signal or sensor data. This mimics how biological neurons fire in response to stimuli. The spikes, which carry information, are recorded temporally (via timing and sequence) rather than in continuous values. SNPs ignore the ever-present noise altogether, much as your brain ignores the sensations of the shirt on your back.

SNPs excel at real-time processing of sensory data, such as audio, visual, or tactile inputs, with minimal latency. We see an example of a song being recognized by its audio signature at the sensor. This makes SNPs suited for the recognition of complex data patterns more effectively, particularly in environments where data is sparse and episodic.

Living on the Edge

Innatera’s SNP’s are fundamentally different and more efficient for specific tasks compared to the GPU-based architectures used for large language models (LLMs) that, by comparison, consume unnecessary energy since they are always on and always processing.

Innatera’s chips are designed to work on the edge, in remote areas where power is scarce and what little power is needed can be supplied by batteries.

Edge processing not only saves the energy typically required to transmit data from the field to the data center, but it also eliminates latency, delivering output for AI systems without pause. Neural processors, because of their energy requirements, require them to be plugged in and, therefore, can be far from sensors. This causes latency problems.

The Innatera T1 SNP SoC requires less than 10 mW of power. Being such a little drain on a battery lets a remote sensor operate for months.

Innatera showed applications of the use. There was a people counter by which an FIR (far infrared sensor) backed by an SNP was able to process the thermal shapes of people in the room. In another room, a radar system was able to filter out the leaves of a houseplant being blown around by a fan. Both applications ensure privacy and minimize power use as they process entirely on the device, and images are of low resolution, unlike vision-based systems.

Innatera vs NVIDIA and Unfair Comparison

NVIDIA neural processors differ significantly from Innatera spiking neural processors. Nvidia’s A100 and H100, for example, are general-purpose AI processors optimized for matrix computations. NVIDIA has explored neuromorphic computing but focuses on simulating SNPs on GPUs rather than building dedicated neuromorphic chips. Compared to Innatera chips, NVIDIA GPUs are larger and consume much more power (100–700 W) than Innatera SNPs.

And then there is the cost. NVIDIA’s H100 costs roughly $25,000 each. (Innatera has not yet released pricing for the T1 SNP).

The Competition

NVIDIA offers low-power chips meant for edge use, such as their Nano and Xavier. Still, with power consumption of 10W and 30W, respectively, there is no comparison to the Innatera’s sub-milliwatt chip.

Innatera is not alone in making spiking neural network chips for edge devices, but a quick survey of the competition showed Innatera to be among the smallest and most energy efficient.

Intel has the Loihi, which operates at relatively low power for its capabilities (~50 mW per chip under typical workloads. IBM’s TrueNorth, with its simulation of a million neurons, a large form factor, and 70 mW power consumption, is better deployed in a data center or research applications than on the edge.

GrAI Matter Labs GrAI One chips are larger than Innatera’s chips as they prioritize computational capabilities for edge-cloud hybrid systems and are more power-hungry, requiring a slightly higher power level (10–50 mW)

SynSense chips are compact and tailored for real-time sensory processing, making them similar in size to Innatera’s chips. Their Dynap-SE consumes around one mW for sensory processing tasks, comparable to Innatera’s chips. SynSense emphasizes vision-based tasks, while Innatera is versatile across sensory modalities like sound, motion, and biological signals.

BrainChip’s Akida AKD100 “reference chip,” which uses Akida’s “neuron fabric” may be Innatera’s closest competition.

About Innatera Nanosystems

I wrap up the meeting with a brief chat with Dr Sumeet Kumar. Dr Kumar earned both his master’s and PhD in microelectronics from Delft University of Technology in the Netherlands. He went on to manage research programs at Delft aimed at developing next-generation compute hardware for highly automated vehicles. Dr Kumar worked at Intel, focusing on developing domain-specific tools for complex media processor architectures. He founded Innatera in 2018.

Dr Kumar is a gentleman, pleasant, well dressed and exudes an air of quiet confidence. We expect someone who thinks on the scale of microns but Dr Kumar thinks of the whole world. He explains how SNPs like Innatera’s, small as they are, could affect climate change. By simply ignoring the torrent of noise from sensors and processing only spikes at the edge, current data centers would be relieved and produce less heat. Data centers working at less than full capacity would obviate the need for more data centers.

The Dutch-based Innatera Nanosystems was spun out of Delft University of Technology in 2018 to develop ultra-low power neuromorphic processors that mimic the brain’s mechanisms for processing sensory data. The company is headquartered in Rijswijk, Netherlands and maintains a design center in Bangalore, India.

Innatera has over 80 employees. It is a private company and, as such, is not required to disclose revenue.

Innatera raised a total of €25 million in funding over two rounds. In November 2020, the company secured €5 million in seed funding. This round was led by Munich-based deep-tech investors MIG Verwaltungs AG and the Industrial Technologies Fund of btov.

In 2024, Innatera raised an additional €20 million in an oversubscribed Series A funding round, including investments from Invest-NL Deep Tech Fund, the EIC Fund, MIG Capital, and Matterwave Ventures.