A Real-Time Simulation Environment for Autonomous Vehicles in Highly Dynamic Driving Scenarios

In May 2018, a group of researchers from the Technical University of Munich (TUM) won the first Roborace Human + Machine Challenge. TUM's autonomous driving software stack manages environment perception, autonomous navigation, and trajectory tracking.

The ability to virtually test the complete autonomous driving software stack, while relying on high-fidelity simulations of the vehicle and its surroundings, is of major importance whenever developing autonomous driving systems. This talk presents a hardware-in-the-loop (HIL) environment based on scalable and expandable hardware that leverages an integrated software solution.

The full autonomous driving stack is simulated in two separate hardware target computers, which mimic the real technical setup of the Robocar—an NVIDIA^® Drive™ PX2 and a Speedgoat real-time target machine. The Mobile real-time target machine, designed specifically to work with Simulink Real-Time™, acts as the vehicle ECU, translating the medium-term desired trajectories into immediate commands for the vehicle actuators though real-time CAN controllers. The NVIDIA Drive PX2 is responsible for tasks such as trajectory planning and sensor processing. Communication between both units is handled by real-time UDP.

A second Speedgoat target machine is used to simulate the vehicle dynamics as a reaction to the vehicle ECU's inputs. This real-time simulator also features sensor and actuator emulators, which make the software think it is operating in the real world with realistic data streams. The physical and behavioral modeling is handled with Simulink^® and Vehicle Dynamics Blockset™, with Simulink Real-Time again enabling the fast prototyping onto the real-time target.

An additional GPU server implements the environment model of the racetrack while providing a full 3D visualization. A twin representation of the real-world racetrack can be easily built using the Level Editor in the Unreal Engine^® by importing track data captured from the vehicle sensors.