What is multispectral imaging technology

Foreword

Multispectral imaging is a method for acquiring and analyzing image data from different spectral bands. Unlike traditional color images, multispectral imaging is able to capture information in a wider spectral range, including bands beyond visible light, such as infrared and ultraviolet light. These different bands correspond to different wavelength ranges, and different substances and materials reflect, absorb, or transmit light in different ways in these bands.

Multispectral cameras use multiple optical sensors or optical filters to separate light of different wavelengths and capture images of each band at the same time, making them camera devices that capture spectral information in different wavelength ranges. It is different from ordinary RGB cameras, which can only capture images in the visible light band, while multispectral cameras can capture a wider spectral range, usually including visible light, infrared, and ultraviolet bands. This enables multispectral cameras to provide richer information than ordinary RGB cameras, and is particularly suitable for many application areas, including agricultural product sorting, farmland inspection, food safety, environmental monitoring, etc.

The development of multispectral cameras

In the 1960s, a new detection technology, multispectral imaging technology, emerged. This technology can obtain spatial image information of targets in different spectral bands at the same time, and can combine imaging technology with spectral technology through the design of optical systems.

Ordinary aerial film cameras used in the early days could only image a specific single spectral band, but could not carry target information. The developed multispectral camera can perform multispectral and multispectral imaging, and the imaging method mainly relies on the filtering effect of bandpass filters. By combining filters, the information carried by the same target in different bands can be received at the same time, thereby realizing imaging in a wide spectral range. Multispectral cameras can be divided into prism splitting structure, filter wheel structure, and filter splitting structure according to different splitting methods.

Classification of multispectral cameras

Prism spectrometry

Prism spectroscopic multispectral cameras typically include an input optical system to direct the incident light. This system may include lenses or other optical components to focus the light onto a prism. The prism beam splitter is the core component of the camera, which is used to disperse the incident light into a spectrum of different wavelengths. Typically, the camera uses one or more prisms, each corresponding to a band. Multiple prisms can be connected in series to disperse multiple bands of light at once. By separating the light of different bands through the prism, and then the separated light enters different cameras for sampling, images of multiple spectra can be obtained.

Advantages:

    High frame rate: very important for applications with high temporal resolution, such as monitoring of dynamic processes

    Full resolution: can capture all bands within a continuous wavelength range, non-discrete energy

    No loss: works based on the principle of refraction and dispersion, does not reduce the intensity of light

Disadvantages:

    High cost: the cost of adjusting optical components and optical paths is extremely high

    Large size: multispectral cameras based on prism spectrometry usually require large prisms and optical components, making the camera too large

Filter wheel technology

Filter wheel cameras use filters to rotate to obtain multi-channel spectral images. These filters are usually located in front of the sensor or lens. This filter wheel usually supports 8-12 bands, each corresponding to a different spectral range. Subsequently, the spectral reflectance of each pixel can be estimated by processing the multispectral image. One of the advantages of filter wheel cameras is that they have full spatial resolution in each band, while allowing filters to be customized and replaced according to the needs of specific applications. However, this camera needs to switch between different bands continuously, and the imaging speed is slow. Therefore, it is only suitable for shooting stationary targets.

Filter Array

Multispectral cameras based on filter arrays can obtain multispectral images in one shot without increasing size or cost. They can usually support multiple channels of visible light, near infrared, and short-wave infrared. It has a wide range of applications in agriculture, environmental monitoring, remote sensing, and satellite imagery. However, due to the limited number of filters on the filter array, the bands covered by this camera are limited.

Multispectral camera technology

Human vision is trichromatic, meaning that each color seen is the product of signals generated by three types of photoreceptor cells located on our retina, a feature that limits our vision to a three-dimensional color space. Now imagine you had a device, like a cell phone, that allowed you to expand your vision into a high-dimensional color space and think about all the hidden information this could reveal. So one way to achieve this is to use multispectral imaging. A series of narrowband images of an object are acquired and then combined into what is called a "spectral cube". This cube therefore holds a lot of information about the object and allows for spectral analysis on each object, so the question is, how do we get this narrowband image?

When light passes through several surfaces with anti-reflection coatings, it resonates and interferes in the gaps that separate these surfaces, resulting in a narrowband transmission spectrum of this structure. Now, if we simply change the optical gap in this filter, this transmission spectrum will shift. The transmission peak will then shift to the infrared range.

As early as 1987, Melonson realized this principle using MEMS devices. However, current MEMS-based Fabry-Perot filters are limited in that their tunable range is very limited, and they can only reduce the initial gap by one-third. When it exceeds one-third, a pulling phenomenon will occur. Multispectral imaging requires a very wide filter tunable range, and the pulling phenomenon can be avoided by simply decoupling optics and MEMS devices.

In this design, we have a movable mirror with a set of external electrodes. The picture below is a physical picture of the tunable filter. It is only 1.05mm thick and consists of three wafers. In this concept, when we apply a driving voltage, the optical gap no longer decreases but increases, and this design can achieve the gap expansion of 6 times.

This bottom-up compatible multispectral camera for mobile phones has been widely used in agricultural inspection, autonomous driving, industrial automation, facial recognition, medicine, etc. Through testing, it can operate normally in a wide range of temperature and pressure, far exceeding the tolerance standards of mobile phones.