Future Electronics — Sensor-Based Machine Learning

Read this to find out about:

• The cost and productivity benefits of predictive maintenance for cobots
• How the application of machine learning techniques to sensor data can detect the early warning signs of machine failure
• Getting started with the Machine Learning Core function in certain industrial sensors from STMicroelectronics

One of the results of the maturing of technologies such as the IoT, augmented reality (AR) and cloud computing is the rise of the smart factory. An increasingly familiar sight in the smart factory is the co-operative robot, or cobot. Cobots already play an important part in smart manufacturing, and are set to take on more functions and provide greater value in the factory as time goes on.

As smart factories come to depend more on cobots for their basic functioning, so it becomes more important that they operate reliably, with no unplanned downtime. This is leading the manufacturers of cobots to enable predictive maintenance in their products: this gives users an early warning of emerging faults which could eventually impair the operation of the cobot. It provides the opportunity to fix faults in planned maintenance time without the disruption caused by an unexpected machine failure.

In a cobot, predictive maintenance systems rely on sensors which can detect small abnormalities in the motion of limbs and joints, and in the motors which drive them:

• Sensors such as accelerometers and inertial measurement units (IMUs) can detect the vibrations caused, for instance, by wear in bearings
• Excessive friction may be detected by distinctive sonic signatures picked up by an ultrasound sensor

The application of machine learning technology, a branch of artificial intelligence (AI), enables the cobot to detect divergence in the patterns of vibration and sound from a reference point set when the cobot was new or in a known undamaged condition. Analysis of anomalous patterns can enable the system to diagnose incipient faults, and to trigger a request to the factory management system to plan repair and maintenance.

In early implementations of machine learning, the complex neural networking algorithms by which patterns in sensor signals are recognized would typically run remotely in an embedded computing system based on a powerful microprocessor.

Such a centralized system, however, processing inputs from a large fleet of cobots, places a heavy burden on the processing infrastructure, resulting in high power consumption, and occupying a large amount of the bandwidth in the network which connects cobots to the central control system.

Now the emergence of a new generation of sensors with embedded AI capabilities offers cobot manufacturers a new way to implement local machine learning. And using tools and software provided by STMicroelectronics, a pioneer in the development of machine learning sensors, cobot design engineers can take advantage of a new, simpler way to build predictive maintenance capability into their products.

Broad range of MEMS sensors for vibration and ultrasound measurement

ST offers one of the industry’s largest portfolios of MEMS sensors, including accelerometers, IMUs, pressure sensors, and microphones. The sensing elements are manufactured using specialized micromachining processes, while the IC interfaces are developed with specialist CMOS technology. This enables the design of a dedicated circuit which is trimmed to match the characteristics of the sensing element.

This technology underpins the high performance of the IIS3DWB, for instance, a three-axis ultra wide-bandwidth MEMS accelerometer, an ideal device for detecting the vibrations generated by a faulty machine. ST also supplies motion sensor modules based on its MEMS sensor ICs: for instance, the ISM330DHCX is a system-in-package featuring a high-performance 3D digital accelerometer and 3D digital gyroscope tailored for Industry 4.0 applications.

Machine learning based on decision tree logic

The ISM330DHCX is one of the ST MEMS sensor products which includes embedded AI capability in the form of a Machine Learning Core (MLC). This machine learning capability allows the system operator to shift some predictive maintenance algorithms from a central application processor to the sensor, where the dedicated MLC consumes much less power.

So how does a sensor’s small, low-power processing logic block provide the machine learning capability which normally calls for a large, power-hungry applications processor? 

The answer is in the decision tree logic which ST embeds in its intelligent sensors: the decision tree algorithms which ST supports are simpler and so consume far fewer instruction cycles, and power, than conventional neural networking algorithms.

A decision tree is a mathematical tool composed of a series of configurable nodes. Each node represents an ‘if-then-else’ condition, for which an input signal, that is, the quantitative values calculated from the raw sensor data, is compared to a threshold value.

The ISM330DHCX can be configured to run up to eight decision trees simultaneously and independently. The decision trees are stored in the device and generate results in the dedicated output registers. The results of the decision tree can be read by a host microcontroller or application processor at any time. The sensor can also generate an interrupt for every change in the result produced by the decision tree.

How decision tree logic operates

The decision tree’s predictive model is built from a set of training data, and is stored in the ISM330DHCX. The training data are logged during the operation of the cobot in its desired state, that is, when in good condition, with no faults.

The decision tree is the method by which the MLC analyzes common features in raw sensor data. These common features will form the basis of the ‘model’ with which the sensor will compare the cobot’s operations over time. If the sensor outputs match the model with a high degree of certainty, then the cobot is fault-free. If the sensor cannot match its real-time measurements to the model, it indicates a potential fault, triggering an alert to the machine operator.

Each node of the decision tree contains a condition in which a feature is compared against a certain threshold. If the condition is true, the next node in the true path is evaluated. If the condition is false, the next node in the false path is evaluated, as shown in Figure 1. The status of the decision tree will evolve node by node until a result is found. The result of the decision tree defines a ‘class’ of behavior: in a fitness wristband, such a class might be ‘walking’ or ‘jogging’. In a cobot’s predictive maintenance application, different classes appropriate to the workload of the cobot apply.