Visualize Sensor Coverage, Detections, and Tracks
Configure and use a Bird's-Eye Plot to display sensor coverage, detections and tracking results around the ego vehicle.
Displaying data recorded in vehicle coordinates on a 2-dimensional map around the ego vehicle is an important part of analyzing sensor coverages, detections and tracking results. Use
to display a snapshot of this information for a certain time or to stream data and efficiently update the display.
This example reads pre-recorded sensor data and tracking results. It includes the following:
Positions, velocities, covariance matrices, and labels of the tracks
Most important object
The above information was recorded at a high rate of 20 updates per second, except vision detections that were recorded at 10 updates per second.
A sensor configuration file defines the position and coverage areas of a vision sensor and a radar sensor with two coverage modes. These coverage areas will be displayed on the bird's-eye plot.
Note that the
birdsEyePlot object sets up a very specific vehicle coordinate system, where the X-axis points forward from the vehicle, the Y-axis points to the left of the vehicle, and the Z-axis points up from the ground. The origin of the coordinate system is typically defined as the center of the rear axle, and the positions of the sensors are defined relative to the origin. For more details, see Coordinate Systems in Automated Driving Toolbox.
Defining Scene Limits and Sensor Coverage
Configuring a bird's-eye plot takes two steps. In the first step, the bird's-eye plot is created, which sets up the coordinate system described above, where the x-axis is directed upwards and y-axis is directed to the left. It is possible to define the axes limits in each direction. In this forward looking example, we define the scene up to 90 meters in front of the ego vehicle and 35 meters on each side.
% Create a bird's-eye plot and limit its axes BEP = birdsEyePlot('Xlimits', [0 90], 'Ylimits', [-35 35]);
In the second step, the bird's-eye plotters are created. The bird's-eye plot offers the following variety of plotters, each configured for plotting a specific data type. They include:
coverageAreaPlotter - Plot sensor coverage areas
detectionPlotter - Plot object detections
trackPlotter - Plot tracks, track uncertainties, and history trails
laneBoundaryPlotter - Plot lane boundaries
pathPlotter - Plot object trajectory
% Create a coverageAreaPlotter for a vision sensor and two radar modes cap(1) = coverageAreaPlotter(BEP,'FaceColor','blue','EdgeColor','blue'); cap(2) = coverageAreaPlotter(BEP,'FaceColor','red','EdgeColor','red'); cap(3) = coverageAreaPlotter(BEP,'FaceColor','red','EdgeColor','red');
Load sensor configuration data. Sensor configuration includes:
The position of the sensors relative to the axes origin (X,Y), in meters
The sensor range, in meters
The sensor yaw angle relative to the x-axis, in degrees
The sensor field of view (FOV), in degrees
load('SensorConfigurationData.mat'); % Use the sensor configuration to plot the sensor coverage areas. Vision % sensor uses the shaded blue coverage area and radar modes are shaded in % red. for i = 1:3 plotCoverageArea(cap(i), [sensorParams(i).X, sensorParams(i).Y],... sensorParams(i).Range, sensorParams(i).YawAngle, sensorParams(i).FoV); end % Add title title('Bird''s-Eye Plot')
The display above shows the coverage of the vision sensor and two radar sensor modes.
The vision sensor is positioned 3.30 meters in front of the origin (rear axle) at the center of the car, with a range of 150 meters and a FOV of 38 degrees.
The radar is positioned 3.38 meters in front of the origin at the center of the car. The radar long-range mode has a range of 174 meters and a FOV of 20 degrees, while the medium-range mode has a range of 60 meters and a FOV of 90 degrees. Note that the coverage areas are truncated at 90 meters in front of the ego vehicle and 35 meters on each side.
This example shows a forward looking scenario; however, you can define coverage area in around the ego vehicle. For example, a sensor that covers from the rear of the vehicle backwards would be oriented with a yaw angle of .
The next few lines read the recorded data in preparation for the next steps.
% Load recorded data from a file load('BirdsEyePlotExampleData.mat', 'dataToDisplay'); % Skip to the 125th time step, where there are 5 vision detections and % multiple radar objects and tracks. timeStep = 125; % Extract the various data from the recorded file for that time step [visionObjectsPos, radarObjectsPos, laneBoundaries, trackPositions, ... trackVelocities, trackCovariances, trackLabels, MIOlabel, MIOposition, ... MIOvelocity] = readDataFrame(dataToDisplay(timeStep));
Next, create plotters to display the recorded vision and radar detections
% create a vision detection plotter put it in a struct for future use bepPlotters.Vision = detectionPlotter(BEP, 'DisplayName','vision detection', ... 'MarkerEdgeColor','blue', 'Marker','^'); % Combine all radar detections into one entry and store it for later update bepPlotters.Radar = detectionPlotter(BEP, 'DisplayName','radar detection', ... 'MarkerEdgeColor','red'); % Call the vision detections plotter plotDetection(bepPlotters.Vision, visionObjectsPos); % Repeat the above for radar detections plotDetection(bepPlotters.Radar, radarObjectsPos);
Plotting Tracks and Most-Important Objects
When adding the tracks to the Bird's-Eye Plot, we provide position, velocity and position covariance information. The plotter takes care of displaying the track history trail, but since this is a single frame, there will be no history.
% Create a track plotter that shows the last 10 track updates bepPlotters.Track = trackPlotter(BEP, 'DisplayName','tracked object', ... 'HistoryDepth',10); % Create a track plotter to plot the most important object bepPlotters.MIO = trackPlotter(BEP, 'DisplayName','most important object', ... 'MarkerFaceColor','black'); % Call the track plotter to plot all the tracks plotTrack(bepPlotters.Track, trackPositions, trackVelocities, trackCovariances, trackLabels); % Repeat for the Most Important Object (MIO) plotTrack(bepPlotters.MIO, MIOposition, MIOvelocity, MIOlabel);
Plotting the Lane Boundaries
Plotting lane boundaries can utilize the
object. To use it, we saved the lane boundaries as parabolicLaneBoundary objects, and call the plotter with it.
% Create a plotter for lane boundaries bepPlotters.LaneBoundary = laneBoundaryPlotter(BEP, ... 'DisplayName','lane boundaries', 'Color',[.9 .9 0]); % Call the lane boundaries plotter plotLaneBoundary(bepPlotters.LaneBoundary, laneBoundaries);
Displaying a Scenario from a Recording File
The recording file contains time-dependent sensor detections, tracking information, and lane boundaries. The next code shows how to play back the recordings and display the results on the bird's-eye plot that was configured above.
Note: vision detections were provided every other frame. In such cases, it is beneficial to show the lack of new sensor detections. To do that, simply pass an empty array to the appropriate plotter to delete the previous detections from the display.
% Rewind to the beginning of the recording file timeStep = 0; numSteps = numel(dataToDisplay); % Number of steps in the scenario % Loop through the scenario as long as the bird's eye plot is open while timeStep < numSteps && isvalid(BEP.Parent) % Promote the timeStep timeStep = timeStep + 1; % Capture the current time for a realistic display rate tic; % Read the data for that time step [visionObjectsPos, radarObjectsPos, laneBoundaries, trackPositions, ... trackVelocities, trackCovariances, trackLabels, MIOlabel, MIOposition, ... MIOvelocity] = readDataFrame(dataToDisplay(timeStep)); % Plot detections plotDetection(bepPlotters.Vision, visionObjectsPos); plotDetection(bepPlotters.Radar, radarObjectsPos); % Plot tracks and MIO plotTrack(bepPlotters.Track, trackPositions, trackVelocities, trackCovariances, trackLabels); plotTrack(bepPlotters.MIO, MIOposition, MIOvelocity, MIOlabel); % Plot lane boundaries plotLaneBoundary(bepPlotters.LaneBoundary, laneBoundaries); % The recorded data was obtained at a rate of 20 frames per second. % Pause for 50 milliseconds for a more realistic display rate. You % would not need this when you process data and form tracks in this % loop. pause(0.05 - toc) end
This example demonstrated how to configure and use a bird's-eye plot object and some of the various plotters associated with it.
Try using the track and most-important object plotters or using the bird's-eye plot with a different recording file.
readDataFrame - extracts the separate fields from the data provided in dataFrame
function [visionObjectsPos, radarObjectsPos, laneBoundaries, trackPositions, ... trackVelocities, trackCovariances, trackLabels, MIOlabel, MIOposition, ... MIOvelocity] = readDataFrame(dataFrame) visionObjectsPos = dataFrame.visionObjectsPos; radarObjectsPos = dataFrame.radarObjectsPos; laneBoundaries = dataFrame.laneBoundaries; trackPositions = dataFrame.trackPositions; trackVelocities = dataFrame.trackVelocities; trackCovariances = dataFrame.trackCovariances; trackLabels = dataFrame.trackLabels; MIOlabel = dataFrame.MIOlabel; MIOposition = dataFrame.MIOposition; MIOvelocity = dataFrame.MIOvelocity; end