Advanced ML for MEMS sensors: Enhancing the accuracy, performance, and power consumption

The current generation of sensors can now collect, analyse, and send important data by utilising Machine Learning (ML) technologies at the edge.

For instance, predictive maintenance leverages ML models to assess sensor data, monitor equipment like motors, and identify early signs of wear or potential failure. This proactive approach helps to prevent breakdowns, reducing downtime and repair expenses.

By incorporating key ML concepts, sensors are now able to process data, extract useful features, and make independent decisions without relying on external computational resources. Essentially, these sensors can analyse data and make predictions without requiring explicit programming.

This article discusses how the advancement of ML algorithms has revolutionised the processing of sensor data. It also emphasises how ML technology has overcome the technical constraints of processing sensor data at the edge.

Smart integration: ML algorithms in MEMS sensors

The integration of ML algorithms into MEMS sensors and AI technology enables the development of a new generation of smart, open, and accurate sensors. This integration reduces the amount of data transferred by the system and offloads network processing, leading to lower power consumption and a more sustainable solution. As a result, precise sensor data provides end users with relevant and actionable information.

MEMS sensors can incorporate various technologies for in-sensor processing, such as sensors with an embedded machine learning core (MLC) and sensors with an intelligent sensor processing unit (ISPU) (see Figure 1). Features like the embedded MLC enable sensors to recognise precise movements and communicate events to a processor with optimal energy efficiency. The integration of ISPU further optimises the necessary computing power, thus maximising system performance.

Sensors with an embedded machine-learning core

The processing capability for sensor data through decision-tree logic is achieved using a decision tree, which is a mathematical tool consisting of configurable nodes defined by an 'if-then-else' condition. These conditions evaluate an input signal, represented by statistical parameters calculated from the sensor data against a threshold.

For example, in STMicroelectronics’ LSM6DSOX system-in-package, this capability is implemented through supervised learning, which involves:

• Identifying classes to be recognised.
• Collecting multiple data logs for each class.
• analysing the collected logs to learn a generic rule that maps inputs (data logs) to outputs (classes).

The classes in an activity recognition algorithm might include stationary, walking, jogging, biking, driving, etc. Multiple data logs must be acquired for each class, such as different people performing the same activity. The analysis of these data logs aims to:

• Define the features needed to classify the different activities accurately.
• Define filters to apply to the input data, enhancing performance using the selected features.
• Generate a dedicated decision tree that recognises one of the classes by mapping inputs to outputs.

Once a decision tree is defined, a provided software tool can generate a device configuration, allowing the decision tree to run on the device, minimising power consumption.

The ML core features inside the LSM6DSOX can be divided into three main blocks (Figure 2):

1. Sensor Data
2. Computation Block
3. Decision Tree

Figure 2: Machine learning core blocks (Source)

Sensor Data:

In this stage the data from the built-in accelerometer, gyroscope, or an additional external sensor connected through the I²C master interface (sensor hub).

Computation Block:

In this stage, filters and features are applied to the input data defined in the first block. Features are statistical parameters computed from the input data (or from the filtered data) within a user-selectable time window. These computed features then act as input for the third block.

Decision Tree:

The Decision Tree assesses the statistical parameters computed in the computation block. It uses a binary tree to compare these parameters against certain thresholds to generate results, such as stationary, walking, jogging, biking, and so on, in the context of activity recognition. An optional 'meta-classifier' filter may also refine the results from the decision tree. The final output of the ML core results includes the decision tree results and any optional meta-classifier filtering.

Inputs:

The input data rate must be equal to or exceed the ML core data rate. For instance, in an activity recognition algorithm running at 26 Hz, the ML core’s Output Data Rate (ODR) must be set to 26 Hz, and the sensor ODRs must be at least 26 Hz or higher.

The ML core uses the following unit conventions:

• Accelerometer data in [g]
• Gyroscope data in [rad/sec]
• External sensor data in [gauss] for a magnetometer

An external sensor, like a magnetometer, can be connected to the LSM6DSOX through the sensor hub feature (mode 2). In this setup, the data from an external sensor can also be used for ML processing, where the first six sensor hub bytes (two per axis) are considered input for the ML core.

Filters:

The basic element of the ML core filtering is a second order IIR filter, as shown in Figure 3.

Figure 3: Filter basic element (Source)

The transfer function of the generic IIR 2nd order filter is,

the outputs can be defined as, from Figure 3,

y(z)=H(z).x(z)

y(z)=y(z). gain

The ML core includes default coefficients for the various filter types (high-pass, band-pass, IIR1, IIR2) to optimise memory usage. After selecting a filter type, the ML core tool helps configure the filter by requesting the necessary coefficients.

Features:

Features refer to the statistical parameters that are derived from the sensor inputs of the machine learning (ML) core. All features are calculated within a specified time window, also known as the 'window length,' expressed as the number of samples. It is important for the user to determine the window size, as it plays a crucial role in ML processing, since all statistical parameters in the decision tree are assessed within this window.

For example, in an activity recognition algorithm, features can be calculated every 2 or 3 seconds. If the sensors are running at 26 Hz, the window length should be around 50 or 75 samples, respectively.

Some ML core features require additional parameters for evaluation, such as a threshold. These features include mean, variance, energy, peak-to-peak, and value.

Mean:

The 'Mean' feature calculates the average of the selected input (I) within the defined time window (WL) using the formula:

Variance:

The 'Variance' feature calculates the variance of the selected input (I) in the defined time window (WL) using this formula:

Energy:

The 'Energy' feature calculates the energy of the selected input (I) in the defined time window (WL) using this formula:

Peak-to-peak:

The 'Peak-to-peak' feature computes the maximum peak-to-peak value of the selected input within the defined time window.

Peak detector:

The 'Peak detector' feature counts the number of peaks (positive and negative) of the selected input within the defined time window.

Decision Tree:

A decision tree is a predictive model created from training data that can be stored in the LSM6DSOX. The training data consists of logs obtained for each class that requires recognition. For example, in activity recognition, the classes may include walking, jogging, and driving.

The inputs to the decision tree are the calculation block results discussed in earlier sections. Each node in the decision tree has a condition that sets a threshold for evaluating a given feature. The next node in the true path is assessed if the condition is met. If not, the subsequent node in the false path is assessed (refer to Figure 3). The decision tree proceeds from node to node until a solution is found.

Figure 4: Decision tree node (Source)

The decision tree produces a new result with each window, which is a user-specified 'window length' for feature computation (Figure 4). The window's length is defined in terms of samples, and the time frame can be calculated by dividing the total number of samples by the data rate used for Machine Learning Core (MLC).

Time window = Window length / MLC_ODR

For example, if the window length is 104 samples and the MLC data rate is 104 Hz, the time window is:

Time window = 104 samples / 104 Hz = 1 second

Sensors with an intelligent sensor processing unit (ISPU)

ST Microelectronics has launched a new line of MEMS devices known as the Intelligent Sensor Processing Unit (ISPU). These devices are suitable for ML-based personal electronics and industrial IoT applications. The ISPU devices integrate sensing and signal conditioning capabilities, along with an ultra-low-power, high-performance programmable core on the same die as the sensor. This core allows for the execution of signal processing and machine learning algorithms within the sensor package.

Understanding intelligent sensor processing unit (ISPU)

The ISPU is an ultra-low power, high-performance programmable DSP designed to process AI algorithms and execute real-time processing directly within the sensor. This technology is aimed at demanding applications at the edge (Figure 5).

Figure 5: Configuration of the intelligent sensor processing unit (Source)

The LSM6DSO16IS and the ISM330IS are products that feature ISPU. The LSM6DSO16IS is designed for consumer applications, while the ISM330IS is intended for industrial applications. Both products contain a 3-axis digital accelerometer and a 3-axis digital gyroscope. These always-on inertial devices have on-die processing capabilities.

ISPU consists of a 32-bit RISC Harvard architecture core, programming and data RAM, and a floating-point unit (FPU) for addition, subtraction, and multiplication. It is capable of connecting and gathering information with external sensors using sensor hub functionality. The core is optimised for real-time execution of machine learning (ML) and deep learning algorithms for processing inertial or external sensor data.

The LSM6DSO16IS is suitable for combining AI with sensors in consumer applications such as gesture recognition, activity recognition, and motion tracking. The ISM330IS is appropriate for AI in edge-based industrial applications like robotics, condition monitoring, and asset tracking.

Sensors with ISPU are highly competitive in power consumption, even when running real-time processing algorithms. By integrating an AI-optimised DSP and a six-axis inertial sensor on the same die, these sensors enable the development of standalone consumer or industrial devices with ultra-low power consumption. The application also requires a host MCU, which can remain in sleep mode and consume little current until woken by the sensor. This approach conserves computational resources, supports edge processing, and prolongs battery life in battery-powered designs.

As a global distributor, we partnered with top suppliers to offer an wide range of sensors and necessary accessories with Machine Learning features:

Conclusion

The STMicroelectronics’ LSM6DSOX demonstrates integration through a decision-tree logic system using supervised learning. This system processes sensor inputs, extracts features, and uses a decision tree to classify activities with minimal power consumption. By leveraging filters and statistical computations within defined time windows, the decision tree efficiently maps inputs to outputs. Advances in ML technology have overcome technical limitations, enhancing the efficiency and capability of edge-based sensor data processing. STMicroelectronics's ISPU devices are best suited for ML-based IoT applications. The LSM6DSO16IS is suitable for consumer applications, while ISM330IS, a 6-axis IMU system-in-package, is designed for industrial applications, both of which are inertial devices with on-die processing. Sensors with ISPU are competitive in power consumption, save computational resources, enable edge processing, and extend battery life.