NVIDIA Goes On Autopilot – Teaches Uber & Volkswagen To Drive Themselves With Pegasus & Xavier

Over the last couple of years, NVIDIA’s push for automotive AI has been, to put it politely, extreme. At pretty much every single press event, it’s making headlines with its DRIVE systems, super power efficient SoCs, and a huge emphasis on AI. At the beginning of CES 2018, today is no different.

NVIDIA’s DRIVE systems are a mesh of linked technologies involving hardware and software, and we were briefed on a number of its upcoming technologies, including some we’ve seen over the course of the last year. Most of it centers around the massive network from different companies, all pledging support for the system. Steadily climbing from 100 companies, now up to 320, all using or developing for NVIDIA’s DRIVE system. The big headlines for this year come from the imminent release of some powerful hardware, and the addition of two big automotive companies that will be making use of DRIVE.

Xavier is the heart of NVIDIA’s big push, an SoC with some impressive technology. It is the most expensive SoC that NVIDIA has made at $2 billion of investment, with the help of 2000 engineers. It’s a custom chip built around a 512-core Volta GPU, hooked up with an 8-core CPU. We know the CPU is an ARM design, but unsure as to its specifications at this time (instruction set, frequency, etc). Additional dedicated hardware is also included on the SoC to help with deep learning (likely Tensor Cores), ‘vision accelerators’, and support for 8K HDR video processing. All this extra hardware will use the same development systems NVIDIA has put in place with existing DRIVE software.

These new Xavier SoCs provide a huge leap in hardware processing power from its previous generation, capable of delivering 30 trillion operations per second on just 30 watts of power – or put another way, it’s 15 times more energy-efficient than the previous generation. We can guess that this is likely down in no small part to the Tensor Core design that NVIDIA integrated into Volta.

Xavier has its own auto-grade board that can be integrated into a new generation of smart cars, and is already being put into production. These boards will enable a number of features that were also being shown at the NVIDIA CES 2018 event in conjunction with Volkswagen. As powerful as Xavier is, it’s not ready for fully autonomous driving, but instead will be used for safer driving or as an ‘intelligent Co-Pilot’.

Xavier will be used for collision detection, brake assistance, driver alertness, terrain mapping and GPS, as well as make use of NVIDIA’s DRIVE IX, or Intelligent Experience platform. This includes facial recognition to unlock the car, gesture control and natural language processing for voice control. One of VW’s first cars to make use of this will be the I.D. Buzz, a fully electric vehicle that’s to be launched around 2020.

While Xavier is a Co-Pilot, if you want something for full autonomy, such as robotaxis that have no need for steering wheels or drivers, then you’ll need to step up to Xavier’s big brother, the DRIVE Pegasus AI platform. It features two Xavier SoCs plus two next-generation GPUs for a massive compute performance of 320 trillion operations per second. This license-plate sized computer is meant to replace a boot full of computers.

Pegasus was shown as early as GTC 2017, a little over 6 months ago, but will begin sampling to the 25 partners NVIDIA has with fully autonomous vehicles, including cars and freight trucks. However, at CES 2018, one company stepped forward to reveal its plans, and that was Uber.

Uber has been a silent partner with NVIDIA using the DRIVE PX system in its self-driving cars since about 2015, and is one of the long-standing 25 companies that NVIDIA likes to bring up regularly, it’s just now, it’s all official. While we won’t see these self-driving cars rolling around quite yet, the speed at which they are being developed is staggering.

Part of the presentation by NVIDIA invloved a number of demos involving how they feed everything back in on itself. Data collected by test cars, the journeys they take, the events that occur, are fed into a simulation, which then builds smarter AIs, which is run on a Xavier system under simulation, then on the hardware. It’s all tested under different weather and lighting conditions, different routes and so forth. At one point, someone joined in the simulation with a ‘bad driver’ and effectively harassed the AI to make it crash, to no effect.

The amount of effort and energy being poured into autonomous driving, the AI systems, and the whole architecture, from software to hardware, is just staggering. The amusing part for NVIDIA? It started off by creating chips to render pretty graphics for games, now it has chips running world simulations using the knowledge to render games, to teach AI how to drive in the real world. Don’t worry though, NVIDIA hasn’t forgotten about us gamers…