Enhancing Image Sensors for Machine Vision

New findings have introduced novel techniques to embed in image sensors to improve their performance in helping machine vision.

FREMONT, CA: Light intensity is measured by image sensors. However, to make significant progress in machine vision, angle, spectrum, and other aspects of light must also be extracted. The latest nanostructured elements incorporated into image sensor chips will likely impact multimodal imaging significantly. These advancements could facilitate autonomous vehicles to see around corners rather than just a straight line, biomedical imaging to identify anomalies at different tissue depths, and telescopes to see through interstellar dust.

Image sensors will slowly experience a transition to emerge as the ideal artificial eyes of machines. An evolution leveraging the capabilities of existing image sensors is likely to have a more immediate impact. Image sensors that convert light into electrical signals comprise millions of pixels on a single chip. The challenge of making multifunctional components as part of the sensor is to combine and miniaturise them.

Researchers have discovered a favourable approach to detecting multiple-band spectra by fabricating an on-chip spectrometer. They placed photonic crystal filters composed of silicon directly on top of the pixels to generate complex interactions between the sensor and incident light. The distribution of light energy is captured by the pixels beneath the films, from which light spectral insights can be deduced. The device is manufactured in less than a hundredth of a square inch. It can be programmed to meet several dynamic ranges, resolution levels, and almost any spectral regime from visible to infrared.

Engineers have come up with a component to detect angular information, determine depth, and construct 3D shapes at subcellular scales. This invention was inspired by directional hearing sensors found in animals such as geckos, who have small heads to recognise where the sound is coming from. Instead, they measure sound direction within a range of orders of magnitude smaller than the corresponding acoustic wavelength using coupled eardrums.

Similarly, pairs of silicon nanowires were constructed as resonators to assist optical resonance. The optical energy in two resonators is sensitive to the incident angle, and the wire nearest to the light sends the most substantial current. The angle of the incoming light waves can be measured by comparing the weakest and strongest currents from both wires. Millions of these nanowires can be placed on a one-square-millimetre chip. This new finding will enhance advances in lensless cameras, augmented reality, and robotic vision.