The superhuman eye: how new color and spectral sensors see what is invisible to people

By Davide Osto

EMEA Vertical Segment Manager (Sensors), Future Electronics

Read this to find out about:

  • The way color sensors mimic the operation of the human eye
  • New flicker detection capabilities in more advanced color sensors
  • How spectral sensors are creating new applications for detailed color analysis, such as testing the readiness of fruit for harvesting

In some fields of electronics, technology can simulate certain human operations without ever matching the rich capabilities of the human species. Artificial intelligence (AI), for instance, has a remarkable ability to perform limited types of thinking, enabling machines to beat humans at a game such as chess. But no-one expects AI to replace human brains for complex operations such as writing a novel, creating a coronavirus vaccine, or designing a building.

 

In the field of optoelectronics, however, sensor technology has developed so far that it now exceeds the capabilities of the eye, our human optical sensor. Driven in large part by the mobile handset industry, the market for light and color sensors has expanded rapidly in the past five years. This has drawn the attention of a growing range of semiconductor manufacturers, and given rise to the development of new types of optical sensor, broadening the range of applications for optical components.

 

These sensors’ capabilities go beyond the powers of the human eye in various ways: some doing what the eye does, but better; others doing what the human eye cannot do at all.

 

In every case, these superhuman capabilities create exciting new possibilities for enhancing existing products’ functions, and for creating completely new types of product.

Like the human eye – but better

The human eye is an RGB (Red/Green/Blue) visible light sensor. The retina contains three types of ‘cones’ which have their peak sensitivity near the red portion of the visible light spectrum at 564 nm to 580 nm, the green at 534 nm to 545 nm, and the blue at 420 nm to 440 nm, as shown in Figure 1. In a human with normal vision, the brain interprets the relative strength of the outputs from these three sets of color sense elements, together with information about light intensity from ‘rods’ in the retina, to construct a fully colored view of the world.

Future Electronics — Machine Vision

Fig. 1: The spectral sensitivity of the average human eye. (Image source: OpenStax College under Creative Commons license)

Many color sensors use this same RGB architecture to enable electronic devices to ‘see’ the world in the same way as a human. A huge number of RGB color sensors are deployed in mobile phone handsets, tablets and laptop computers, providing color sense inputs to enable display brightness and color management. This is the function which dims a handset’s screen when the user moves from bright sunlight outdoors to a dimly lit indoor space, and which adjusts the color balance to take account of changing ambient light conditions.

 

In this type of application, high value is attached to the ability to mimic the way that the human eye responds to external optical stimuli, by a combination of configuring the optical filters over the sensor’s photodiodes to pass light at the specified wavelengths, and applying software to compensate for variances from the 1931 International Commission on Illumination (CiE) standard model of the spectral sensitivity of the human eye.

 

But wonderful as the gift of vision is for normally-sighted humans, there is much the human eye does poorly or not at all. This fact inspired the release in 2021 of an enhanced RGB sensor from STMicroelectronics, the VD6283TX. According to ST, this is the world’s smallest RGB + infrared (IR) sensor, measuring 1.83 mm x 1.0 mm x 0.55 mm. Like the human eye, it is sensitive to RGB wavelengths, though with peak sensitivity in the red portion of the visible light spectrum at a slightly longer wavelength than the eye, as shown in Figure 2.

Future Electronics — Machine Vision

Fig. 2: Spectral sensitivity of the six-channel VD6283TX color sensor

Like the human eye, the VD6283TX is also sensitive to flicker. But while the eye loses its sensitivity to flickering light at a frequency of between 50 Hz and 90 Hz1, and so at higher frequencies sees continuous light, the VD6283TX maintains flicker sensitivity from 100 Hz to 2 kHz. Since the flicker of LED lights generally occurs at between 100 Hz and 500 Hz, the ST sensor can detect all common sources of flickering light. Using the flicker frequency detection signal from the VD6283TX, camera systems can apply countermeasures to eliminate the unwanted banding effects seen on video or still images captured in flickering light.

 

Flicker which is visible to the human eye is uncomfortable to view, and can even be harmful to health, in some people triggering the onset of headaches or nausea. But in fact research has also uncovered similar impacts on well-being caused by non-visible flicker at relatively low frequencies. Use of the VD6283TX enables lighting or building management systems to detect the presence of potentially harmful invisible flicker, and provide an appropriate indication to the occupants.

 

The high sensitivity of the VD6283TX photodiodes means that it can sustain flicker detection capability in dim light conditions down to just 1 lux. And it does not only detect flicker, it provides a measurement of the flicker frequency which is accurate to ±3%.

 

The VD6283TX can easily be evaluated with a stand-alone board, the X-NUCLEO-6283A1, or as part of a development system using the P-NUCLEO-6283A1, which includes an STM32F401 microcontroller board.

Beyond the human eye capabilities

As a three-channel sensor, the human eye is a relatively crude hardware device for discriminating between one color and another. In fact, it relies on sophisticated ‘software’, the brain’s sensory perception function, for interpreting the raw RGB signal from the eye, and for building a spectral analysis of the scene in view.

 

This interpretation function can sometimes be fooled by its input signal. And the brain cannot interpret anything about ultraviolet and infrared light, since the eye is insensitive to wavelengths below around 380 nm and above 750 nm.

 

Now a new generation of optical sensors called spectral sensors are providing a hardware capability which goes beyond the limited functions performed by the eye. Austria-based manufacturer ams OSRAM is a pioneer in this field, thanks to unique technology it has developed for fabricating interferometric filters on-wafer. This manufacturing capability allows it to create spectral sensors with uniform filtering characteristics in a very small form factor, and at low cost through the application of high-volume semiconductor fabrication processes.

Future Electronics — Machine Vision

Fig. 3: Spectral sensitivity of the AS7341 sensor from ams OSRAM

The AS7341 is the first ams OSRAM spectral sensor to achieve widespread adoption across a broad range of applications. This 11-channel device splits the visible light spectrum into eight bands, and also includes an IR channel, a clear channel and a flicker channel, as seen in Figure 3.

 

The more detailed spectral analysis which is enabled by the AS7341’s eight visible light channels, compared to the three of the human eye, has myriad uses in the real world.

 

One is reliable color analysis and color matching, a crucial function in the fashion, cosmetics, furnishings, coatings, textiles, broadcasting, film and other industries. As described above, the human eye can be fooled by the brain, which interprets the signal from the retina. To the human eye, for instance, a mix of red and yellow light appears orange in most lights,  and appears identical to a saturated ‘pure’ orange.

 

Under certain lights, however, a red-yellow object will appear to be a different color from orange, a phenomenon known as metamerism. A handheld spectrometer based on the AS7341 will correctly distinguish a red-yellow object from a pure orange object, avoiding the risk of pairing two samples which appear to be matched under a reference light, but which might be revealed to the end user to be different colors when viewed under a different light.

 

The superior color discrimination of the 11-channel AS7341 also has uses in fields which do not involve color matching. In traditional domestic and commercial smoke alarms, an optical sensor is used to detect when the detection chamber is occupied by smoke, which occludes the view of a photodiode in the chamber. The detector can also be triggered by other contaminants, such as vapor droplets, causing false alarms.

 

By replacing the standard photodiode with an AS7341, the smoke alarm can be substantially upgraded, providing the ability to distinguish smoke from vapor. It can also detect the distinctive spectral ‘signature’ of the smoke of different types of fuel, such as wood, plastic, and oil. An intelligent smoke alarm could alert first responders not only to the existence of the fire, but an indication of the type of fire extinguisher required to douse it.

 

This concept of the spectral signature has many other potential applications: ams OSRAM has demonstrated the use of a chip-scale spectral sensor in performing contactless, non-destructive testing of the ripeness of crops such as kiwi fruit2. This is valuable for many types of produce which are shipped long distances, and which have to be picked before fully ripe in order to arrive at the supermarket shelf in the perfect condition. A spectral analysis of light reflected from the surface of the fruit reveals precise and accurate data about the fruit’s sugar and dry matter content, the data points required to determine readiness for harvesting.

 

Many more such applications are emerging as the industry discovers the functionality of this new generation of spectrometers-on-chip, devices which provide much of the capability of a lab-grade instrument in a device often available at a unit price of less than $10. Even more sophisticated applications will emerge following the introduction of new devices with more than 11 channels, for even more precise characterization of the spectral signatures of objects.

Implementing spectral analysis functions in board-level designs

The new capabilities provided by the latest color and spectral sensors will draw many design engineers into the field of optoelectronics for the first time. The VD6283TX and AS7341 are in many ways like any other board-level component, providing a familiar I2C interface to a host controller. Support from machine vision and optoelectronics specialists at Future Electronics can help first-time users to master the optical elements of the system design, such as selection of a reference light source where necessary, and the selection and application of a lens.

 

Future Electronics can also help first-time users by supplying and supporting evaluation and development kits for the VD6283TX, as well as the AS7341 EVAL KIT from ams OSRAM for the AS7341.

Evaluation Board

Part Number: X-NUCLEO-6283A1 

The X-NUCLEO-6283A1 is an expansion board which is compatible with the STMicroelectronics STM32 Nucleo microcontroller development boards. The board provides a complete kit for developing applications using the VD6283 color sensor. ST supplies full system software, including code examples and a graphical user interface, with the X-NUCLEO-6283A1.

Part Number: AS7341 EVAL KIT 

The AS7341 Evalboard is a platform for evaluating the ams OSRAM AS7341 11-channel spectral sensor. It is supplied with an AS7341 software GUI which runs on the Windows operating system.

Share This

Subscribe to our newsletters

Subscribe to Future Electronics

Get access to the latest product information, technical analysis, design notes and more

Choose your region

FTM Board Club

Sign up for access to exclusive development boards, an essential tool for many innovative design projects.

*Available to pre-qualified EMEA customers only.

Related Articles

ECS Inc — ECX-2236B Crystals
10 November 2022
The ECX-2236B from ECS Inc., is a family of compact, surface-mount crystals which are notable for their...
Read More
STMicroelectronics — BLUENRG-LP Wireless SoC
10 November 2022
Ultra-low power programmable Bluetooth® Low Energy wireless SoC solution by STMicroelectronics with unparalleled...
Read More
I-PEX — MHF Micro-RF Connector Series
22 June 2023
I-PEX MHF Micro-RF Connector Series is the ideal locking solution that work with MHF I receptacles supporting...
Read More