Main Content

multiObjectTracker

Track objects using GNN assignment

Description

The multiObjectTracker System object™ initializes, confirms, predicts, corrects, and deletes the tracks of moving objects. Inputs to the multi-object tracker are detection reports generated by an objectDetection object, radarDetectionGenerator object, or visionDetectionGenerator object. The multi-object tracker accepts detections from multiple sensors and assigns them to tracks using a global nearest neighbor (GNN) criterion. Each detection is assigned to a separate track. If the detection cannot be assigned to any track, based on the AssignmentThreshold property, the tracker creates a new track. The tracks are returned in a structure array.

A new track starts in a tentative state. If enough detections are assigned to a tentative track, its status changes to confirmed. If the detection is a known classification (the ObjectClassID field of the returned track is nonzero), that track can be confirmed immediately. For details on the multi-object tracker properties used to confirm tracks, see Algorithms.

When a track is confirmed, the multi-object tracker considers that track to represent a physical object. If detections are not added to the track within a specifiable number of updates, the track is deleted.

The tracker also estimates the state vector and state vector covariance matrix for each track using a Kalman filter. These state vectors are used to predict a track's location in each frame and determine the likelihood of each detection being assigned to each track.

To track objects using a multi-object tracker:

  1. Create the multiObjectTracker object and set its properties.

  2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?.

Creation

Description

tracker = multiObjectTracker creates a multiObjectTracker System object with default property values.

example

tracker = multiObjectTracker(Name,Value) sets properties for the multi-object tracker using one or more name-value pairs. For example, multiObjectTracker('FilterInitializationFcn',@initcvukf,'MaxNumTracks',100) creates a multi-object tracker that uses a constant-velocity, unscented Kalman filter and maintains a maximum of 100 tracks. Enclose each property name in quotes.

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

Unique tracker identifier, specified as a nonnegative integer. This property is used as the SourceIndex in the tracker outputs, and distinguishes tracks that come from different trackers in a multiple-tracker system. You must specify this property as a positive integer to use the track outputs as inputs to a track fuser.

Example: 1

Kalman filter initialization function, specified as a function handle or as a character vector or string scalar of the name of a valid Kalman filter initialization function.

Automated Driving Toolbox™ supplies several initialization functions that you can use to specify FilterInitializationFcn.

Initialization FunctionFunction Definition
initcvekfInitialize constant-velocity extended Kalman filter.
initcvkfInitialize constant-velocity linear Kalman filter.
initcvukfInitialize constant-velocity unscented Kalman filter.
initcaekfInitialize constant-acceleration extended Kalman filter.
initcakfInitialize constant-acceleration linear Kalman filter.
initcaukf Initialize constant-acceleration unscented Kalman filter.
initctekf Initialize constant-turnrate extended Kalman filter.
initctukfInitialize constant-turnrate unscented Kalman filter.

You can also write your own initialization function. The input to this function must be a detection report created by objectDetection. The output of this function must be a Kalman filter object: trackingKF, trackingEKF, or trackingUKF. To guide you in writing this function, you can examine the details of the supplied functions from within MATLAB®. For example:

type initcvkf

Data Types: function_handle | char | string

Detection assignment threshold (or gating threshold), specified as a positive scalar or an 1-by-2 vector of [C1,C2], where C1C2. If specified as a scalar, the specified value, val, will be expanded to [val, Inf].

Initially, the tracker executes a coarse estimation for the normalized distance between all the tracks and detections. The tracker only calculates the accurate normalized distance for the combinations whose coarse normalized distance is less than C2. Also, the tracker can only assign a detection to a track if their accurate normalized distance is less than C1. See the distance function used with tracking filters (for example, trackingEKF) for an explanation of the distance calculation.

Tips:

  • Increase the value of C2 if there are combinations of track and detection that should be calculated for assignment but are not. Decrease it if cost calculation takes too much time.

  • Increase the value of C1 if there are detections that should be assigned to tracks but are not. Decrease it if there are detections that are assigned to tracks they should not be assigned to (too far away).

Maximum number of tracks that the tracker can maintain, specified as a positive integer.

Data Types: double

Maximum number of sensors that can be connected to the tracker, specified as a positive integer. When you specify detections as input to the multi-object tracker, MaxNumSensors must be greater than or equal to the highest SensorIndex value in the detections cell array of objectDetection objects used to update the multi-object tracker. This property determines how many sets of ObjectAttributes fields each output track can have.

Data Types: double

Threshold for track confirmation, specified as a two-element vector of non-decreasing positive integers, [M N], where M is less than or equal to N. A track is confirmed if it receives at least M detections in the last N updates.

  • When setting M, take into account the probability of object detection for the sensors. The probability of detection depends on factors such as occlusion or clutter. You can reduce M when tracks fail to be confirmed or increase M when too many false detections are assigned to tracks.

  • When setting N, consider the number of times you want the tracker to update before it makes a confirmation decision. For example, if a tracker updates every 0.05 seconds, and you allow 0.5 seconds to make a confirmation decision, set N = 10.

Example: [3 5]

Data Types: double

Threshold for track deletion, specified as a two-element vector of positive non-decreasing integers [P Q], where P is less than or equal to Q. If a confirmed track is not assigned to any detection P times in the last Q tracker updates, then the track is deleted.

  • Decrease Q (or increase P) if tracks should be deleted earlier.

  • Increase Q (or decrease P) if tracks should be kept for a longer time before deletion.

Example: [3 5]

Data Types: single | double

Enable a cost matrix as input to the multiObjectTracker System object or to the updateTracks function, specified as false or true.

Data Types: logical

Enable the input of detectable track IDs at each object update, specified as false or true. Set this property to true if you want to provide a list of detectable track IDs. This list tells the tracker of all tracks that the sensors are expected to detect and, optionally, the probability of detection for each track.

Data Types: logical

Parameters of the track state reference frame, specified as a struct or a struct array. Use this property to define the track state reference frame and how to transform the track from the tracker (called source) coordinate system to the fuser coordinate system.

This property is tunable.

Data Types: struct

This property is read-only.

Number of tracks maintained by the multi-object tracker, specified as a nonnegative integer.

Data Types: double

This property is read-only.

Number of confirmed tracks, specified as a nonnegative integer. The IsConfirmed fields of the output track structures indicate which tracks are confirmed.

Data Types: double

Usage

To update tracks, call the created multi-object tracker with arguments, as if it were a function (described here). Alternatively, update tracks by using the updateTracks function, specifying the multi-object tracker as an input argument.

Description

confirmedTracks = tracker(detections,time) creates, updates, and deletes tracks in the multi-object tracker and returns details about the confirmed tracks. Updates are based on the specified list of detections, and all tracks are updated to the specified time. Each element in the returned confirmedTracks corresponds to a single track.

example

[confirmedTracks,tentativeTracks] = tracker(detections,time) also returns tentativeTracks containing details about the tentative tracks.

example

[confirmedTracks,tentativeTracks,allTracks] = tracker(detections,time) also returns allTracks containing details about all the confirmed and tentative tracks. The tracks are returned in the order by which the tracker internally maintains them. You can use this output to help you calculate the cost matrix, an optional input argument.

[___] = tracker(detections,time,costMatrix) specifies a cost matrix, returning any of the outputs from preceding syntaxes.

To specify a cost matrix, set the HasCostMatrixInput property of the multiObjectTracker System object to true.

[___] = tracker(___,detectableTrackIDs) also specifies a list of expected detectable tracks given by detectableTrackIDs. This argument can be used with any of the previous input syntaxes.

To enable this syntax, set the HasDetectableTrackIDsInput property to true.

Input Arguments

expand all

Detection list, specified as a cell array of objectDetection objects. The Time property value of each objectDetection object must be less than or equal to the current time of update, time, and greater than the previous time value used to update the multi-object tracker.

Time of update, specified as a real scalar. The multi-object tracker updates all tracks to this time. Units are in seconds.

time must be greater than or equal to the largest Time property value of the objectDetection objects in the input detections list. time must increase in value with each update to the multi-object tracker.

Data Types: double

Cost matrix, specified as a real-valued NT-by-ND matrix, where NT is the number of existing tracks, and ND is the number of current detections. The rows of the cost matrix correspond to the existing tracks. The columns correspond to the detections. Tracks are ordered as they appear in the list of tracks in the allTracks output argument of the previous update to the multi-object tracker.

In the first update to the multi-object tracker, or when the multi-object tracker has no previous tracks, assign the cost matrix a size of [0, ND]. The cost must be calculated so that lower costs indicate a higher likelihood that the multi-object tracker assigns a detection to a track. To prevent certain detections from being assigned to certain tracks, use Inf.

Dependencies

To enable specification of the cost matrix when updating tracks, set the HasCostMatrixInput property of the multi-object tracker to true

Data Types: double

Detectable track IDs, specified as a real-valued M-by-1 vector or M-by-2 matrix. Detectable tracks are tracks that the sensors expect to detect. The first column of the matrix contains a list of track IDs that the sensors report as detectable. The optional second column contains the detection probability for the track. The detection probability is either reported by a sensor or, if not reported, obtained from the DetectionProbability property.

Tracks whose identifiers are not included in detectableTrackIDs are considered as undetectable. The track deletion logic does not count the lack of detection as a 'missed detection' for track deletion purposes.

Dependencies

To enable this input argument, set the detectableTrackIDs property to true.

Data Types: single | double

Output Arguments

expand all

Confirmed tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack.

A track is confirmed if it satisfies the confirmation threshold specified in the ConfirmationThreshold property. In that case, the IsConfirmed property of the object or field of the structure is true.

Data Types: struct | object

Tentative tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack.

A track is tentative if it does not satisfy the confirmation threshold specified in the ConfirmationThreshold property. In that case, the IsConfirmed property of the object or field of the structure is false.

Data Types: struct | object

All tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack. All tracks consists of confirmed and tentative tracks.

Data Types: struct | object

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

updateTracksUpdate multi-object tracker with new detections
initializeTrackInitialize new track
deleteTrackDelete existing track
getTrackFilterPropertiesObtain filter properties of track from multi-object tracker
setTrackFilterPropertiesSet filter properties of track from multi-object tracker
predictTracksToTimePredict track state
stepRun System object algorithm
releaseRelease resources and allow changes to System object property values and input characteristics
cloneCreate duplicate System object
isLockedDetermine if System object is in use
resetReset internal states of System object

Examples

collapse all

Create a multiObjectTracker System object™ using the default filter initialization function for a 2-D constant-velocity model. For this motion model, the state vector is [x;vx;y;vy].

tracker = multiObjectTracker('ConfirmationThreshold',[4 5], ...
    'DeletionThreshold',10);

Create a detection by specifying an objectDetection object. To use this detection with the multi-object tracker, enclose the detection in a cell array.

dettime = 1.0;
det = { ...
    objectDetection(dettime,[10; -1], ...
    'SensorIndex',1, ...
    'ObjectAttributes',{'ExampleObject',1}) ...
    };

Update the multi-object tracker with this detection. The time at which you update the multi-object tacker must be greater than or equal to the time at which the object was detected.

updatetime = 1.25;
[confirmedTracks,tentativeTracks,allTracks] = tracker(det,updatetime);

Create another detection of the same object and update the multi-object tracker. The tracker maintains only one track.

dettime = 1.5;
det = { ...
    objectDetection(dettime,[10.1; -1.1], ...
    'SensorIndex',1, ...
    'ObjectAttributes',{'ExampleObject',1}) ...
    };
updatetime = 1.75;
[confirmedTracks,tentativeTracks,allTracks] = tracker(det,updatetime);

Determine whether the track has been verified by checking the number of confirmed tracks.

numConfirmed = tracker.NumConfirmedTracks
numConfirmed = 0

Examine the position and velocity of the tracked object. Because the track has not been confirmed, get the position and velocity from the tentativeTracks structure.

positionSelector = [1 0 0 0; 0 0 1 0];
velocitySelector = [0 1 0 0; 0 0 0 1];
position = getTrackPositions(tentativeTracks,positionSelector)
position = 1×2

   10.1426   -1.1426

velocity = getTrackVelocities(tentativeTracks,velocitySelector)
velocity = 1×2

    0.1852   -0.1852

Create a sequence of detections of a moving object. Track the detections using a multiObjectTracker System object™. Observe how the tracks switch from tentative to confirmed and then to deleted.

Create a multi-object tracker using the initcakf filter initialization function. The tracker models 2-D constant-acceleration motion. For this motion model, the state vector is [x;vx;ax;y;vy;ay].

tracker = multiObjectTracker('FilterInitializationFcn',@initcakf, ...
    'ConfirmationThreshold',[3 4],'DeletionThreshold',[6 6]);

Create a sequence of detections of a moving target using objectDetection. To use these detections with the multiObjectTracker, enclose the detections in a cell array.

dt = 0.1;
pos = [10; -1];
vel = [10; 5];
for detno = 1:2
    time = (detno-1)*dt;
    det = { ...
        objectDetection(time,pos, ...
        'SensorIndex',1, ...
        'ObjectAttributes',{'ExampleObject',1}) ...
        };
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the track has not been confirmed yet by checking the number of confirmed tracks.

numConfirmed = tracker.NumConfirmedTracks
numConfirmed = 0

Because the track is not confirmed, get the position and velocity from the tentativeTracks structure.

positionSelector = [1 0 0 0 0 0; 0 0 0 1 0 0];
velocitySelector = [0 1 0 0 0 0; 0 0 0 0 1 0];
position = getTrackPositions(tentativeTracks,positionSelector)
position = 1×2

   10.6669   -0.6665

velocity = getTrackVelocities(tentativeTracks,velocitySelector)
velocity = 1×2

    3.3473    1.6737

Add more detections to confirm the track.

for detno = 3:5
    time = (detno-1)*dt;
    det = { ...
        objectDetection(time,pos, ...
        'SensorIndex',1, ...
        'ObjectAttributes',{'ExampleObject',1}) ...
        };
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the track has been confirmed, and display the position and velocity vectors for that track.

numConfirmed = tracker.NumConfirmedTracks
numConfirmed = 1
position = getTrackPositions(confirmedTracks,positionSelector)
position = 1×2

   13.8417    0.9208

velocity = getTrackVelocities(confirmedTracks,velocitySelector)
velocity = 1×2

    9.4670    4.7335

Let the tracker run but do not add new detections. The existing track is deleted.

for detno = 6:20
    time = (detno-1)*dt;
    det = {};
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the tracker has no tentative or confirmed tracks.

isempty(allTracks)
ans = logical
   1

Generate detections using a forward-facing automotive radar mounted on an ego vehicle. Assume that there are three targets:

  • Vehicle 1 is in the center lane, directly in front of the ego vehicle, and driving at the same speed.

  • Vehicle 2 is in the left lane and driving faster than the ego vehicle by 12 kilometers per hour.

  • Vehicle 3 is in the right lane and driving slower than the ego vehicle by 5 kilometers per hour.

All positions, velocities, and measurements are relative to the ego vehicle. Run the simulation for ten steps.

dt = 0.1;
pos1 = [150 0 0];
pos2 = [160 10 0];
pos3 = [130 -10 0];
vel1 = [0 0 0];
vel2 = [12*1000/3600 0 0];
vel3 = [-5*1000/3600 0 0];
car1 = struct('ActorID',1,'Position',pos1,'Velocity',vel1);
car2 = struct('ActorID',2,'Position',pos2,'Velocity',vel2);
car3 = struct('ActorID',3,'Position',pos3,'Velocity',vel3);

Create an automotive radar sensor that is offset from the ego vehicle. By default, the sensor location is at (3.4,0) meters from the vehicle center and 0.2 meters above the ground plane. Turn off the range rate computation so that the radar sensor measures position only.

radar = radarDetectionGenerator('DetectionCoordinates','Sensor Cartesian', ...
    'MaxRange',200,'RangeResolution',10,'AzimuthResolution',10, ...
    'FieldOfView',[40 15],'UpdateInterval',dt,'HasRangeRate',false);
tracker = multiObjectTracker('FilterInitializationFcn',@initcvkf, ...
    'ConfirmationThreshold',[3 4],'DeletionThreshold',6);

Generate detections with the radar from the non-ego vehicles. The output detections form a cell array and can be passed directly in to the multiObjectTracker.

simTime = 0;
nsteps = 10;
for k = 1:nsteps
    dets = radar([car1 car2 car3],simTime);
    [confirmedTracks,tentativeTracks,allTracks] = tracker(dets,simTime);

Move the cars one time step and update the multi-object tracker.

    simTime = simTime + dt;
    car1.Position = car1.Position + dt*car1.Velocity;
    car2.Position = car2.Position + dt*car2.Velocity;
    car3.Position = car3.Position + dt*car3.Velocity;
end

Use birdsEyePlot to create an overhead view of the detections. Plot the sensor coverage area. Extract the X and Y positions of the targets by converting the Measurement fields of the cell array into a MATLAB array. Display the detections on the bird's-eye plot.

BEplot = birdsEyePlot('XLim',[0 220],'YLim',[-75 75]);
caPlotter = coverageAreaPlotter(BEplot,'DisplayName','Radar coverage area');
plotCoverageArea(caPlotter,radar.SensorLocation,radar.MaxRange, ...
    radar.Yaw,radar.FieldOfView(1))
detPlotter = detectionPlotter(BEplot,'DisplayName','Radar detections');
detPos = cellfun(@(d)d.Measurement(1:2),dets,'UniformOutput',false);
detPos = cell2mat(detPos')';
if ~isempty(detPos)
    plotDetection(detPlotter,detPos)
end

Algorithms

When you pass detections into a multi-object tracker, the System object:

  • Attempts to assign the input detections to existing tracks, based on the AssignmentThreshold property of the multi-object tracker.

  • Creates new tracks from unassigned detections.

  • Updates already assigned tracks and possibly confirms them, based on the ConfirmationThreshold property of the tracker.

  • Deletes tracks that have no assigned detections, based on the DeletionThreshold property of the tracker.

Compatibility Considerations

expand all

Behavior changed in R2020a

Extended Capabilities

Introduced in R2017a