Qualcomm Introduces Trio Of Cameras To Spectra Module Program For Super-Accurate Depth Sensing

Making smartphones smarter is Qualcomm’s bread and butter, and with the expanding possibilities that are being generated from the continued development of virtual and augmented realities, it’s in Qualcomm’s interest to keep the technologies rolling in. The second generation Spectra Image Signal Processor (ISP) that’ll come with the next Snapdragon SoC offers a number of features that’ll have VR and content creators salivating.

Computer vision and processing has gone through an odd period as of late. While cameras have become ever denser in pixel counts, and the signal processing behind them has improved, going beyond that has been difficult. More recently, depth-sensing technologies and iris scanning for security have been slowly introduced, and Qualcomm has three new camera modules ready to be released.

The first is a front facing camera for iris recognition, used for unlocking the phone and general security. It has a neat trick where it can scan your iris, even through sunglasses, with just 40ms response time. However, it’s the two other cameras that’ll have more interest for our crowd, and that’s the depth sensing modules.

With two different modules, being either passive or active, the Spectra camera can get a feel for its surroundings through either a stereoscopic image that’s then used for perspective analysis, or a premium active depth-sensing system that uses an infra-red emitter through a special dot filter to create 3D maps of objects in front of it. The active system uses less power, and generates a much more precise image than the dual-camera setup (likely due to less complexity with processing the IR map), but the end result really needs to be seen in action to get an idea of its capabilities.

With the video above you can see a pianist being recorded with a prototype system showing off the two camera setups. It’s the 3D reconstruction of the scene that’s most impressive, as it’s able to rebuild part of the scene from just a single perspective, and with a great deal of accuracy. Qualcomm states the system uses 10,000 points of depth and measure them to within 0.125mm of each other – effectively sub-pixel accuracy. This system can even be used for facial recognition as well.

Depth sensing has many applications in the eXtended Reality (XR) space – that’s the mix of virtual and augmented reality. From collision detection to object interaction overlays, there are a great deal of possibilities. We’re likely to start seeing all of this being rolled out, even in part, with Google Daydream and Project Tango.

While not as cool as the 3D reconstruction above, the Spectra ISP has another cool feature up its sleeve for all those that enjoy using their phones to record video. One of the limitations of such a small sensor in a camera is that they have a limit as to how much light they can accurately harvest for a decent image. When you are outdoors, this isn’t too much of a problem, as the Sun provides more than enough light for a decent image, however, it’s when you go inside or try to take movies at night that things become problematic.

When there is not enough light to illuminate a scene, the camera uses digital enhancement to brighten the image, but this results in noise being generated, either as flickering dots or smudges of color in dark areas. A new trick with the Spectra ISP is something called motion compensated temporal filtering (MCTF) and electronic image stabilizing (EIS). The ISP goes through multiple frames of a video to pick out details that may have been caught with less noise, and rebuilds a new frame with greater detail afterwards. Not only will you get less blurring from the image stabilizing, but much better detail in low-light areas.

These features won’t be accessible just yet, as Qualcomm stated they’ll be part of the next Snapdragon platform, which likely means next year some time. However, Qualcomm’s other projects with the Daydream reference design and Asus with the ZenFone AR featuring project Tango will give us some idea about what’s to come.