In the TOMHT tracker, how do I change the state vector and add more filters?

We are working with the TOMHT tracker, and I noticed that the state vector is only (x, y, vx, vy). This seems very limiting, but I can't figure out how to modify this. Is there any documentation on this, or can somebody explain? For example, I would like to add ax and ay to the state vector and implement an additional filter that predicts constant accelaration. Furthermore, Is there a more detailed description on the TOMHT tracker?

Thank you very much,

Joost

Answer 1

[Elad Kivelevitch]

The default state depends on the FilterInitializationFcn property, which is initcvekf. initcvekf initializes a constant velocity (CV) extended Kalman filter (EKF). You can choose from a variety of other init functions by using tab-completion in the MATLAB Window:

trackerTOMHT('FilterInitializationFcn', @init<TAB % <TAB means hit the tab button here

This will give you a wide range of initialization functions. One of them, initcaekf, initializes filters based on the constant acceleration (CA) extended Kalman filter (EKF).

Please use the documentation of each function to know how the state vector is defined and which motion and measurement models are used.

Additionally, You can always create your own initalization functions, and in each one of them you can define the filter to use and which motion and measurement models to use. Please do not feel restricted to use the built-in motion and measurement models or the initialization functions that ship with the product.

[Joost]

Thank you for the fast answer, that is very clear. I have one follow up question. It seems that the TOMHT tracker handles 1 prediction per track. However, one of the filters I can choose is the EKFIMM, which returns an IMM object (with multiple predictions). Whereas the other filters return a filter object. My question is, if I choose the EKFIMM filter, will the TOMHT tracker be able to handle this, and if so, how does he handle this?

Thanks you very much,

Joost Somers

[Elad Kivelevitch]

Hi Joost,

The TOMHT can certainly work with the trackingIMM returned by the initekfimm function.

The initekfimm uses three extended Kalman filters, each with a different motion model: constant velocity, constant acceleration, and constant turn. We handle the switching between state definitions using the switchimm function that you can see in the filter returned by initekfimm. Internally, each filter mainains its own estimate of target state as a normal distribution (like any EKF) and each filte is predicted and corrected separately. The filter estimates are mixed and combined in the way described in the literature for interacting multiple models such that in the end you get a combined estimate of the IMM filter. You can use the TransitionProbabilities to control the Markov process that controls the transition from one model to another. The output of the IMM filter - which is reflected in the tracker output (the tracks) is the combined state and state covariance and it uses the state definition of the first filter, which is CV in the case of initekfimm. If you would like to use the constant acceleration state definition as the state output, I recommend that you open the file initekfimm in the editor, copy its content to another file, and simply switch the filter order such that the first filter is the one initialized by initcaekf and replace what used to be the CA filter with the one returned by initcvekf. Everything else should work based on that.

In the very abstract way, a tracker contains three elements: association of new detections to tracks, track management (initialization, confirmation, deletion), and the filtering. The TOMHT, GNN and JPDA trackers offer different ways to handle the association of new detections to tracks maintained by the tracker. The filtering is done with the filter you choose with your filter initialization function. So, these two are complementary to each other. The TOMHT, similar to the GNN and JPDA trackers, works with all these filters because all the filters adhere to the same abstract tracking filter interface. Therefore, the tracker calls common methods like predict, correct, likelihood, etc. to use the filters without knowing how exactly each one of them implements their algorithm. We designed it this way to allow you the flexibility to choose between filters, motion models, and even trackers and to perform "what-if" analysis. You can see some of that flexibility in the example: https://www.mathworks.com/help/fusion/ug/tracking-closely-spaced-targets-under-ambiguity.html

From your question it was unclear whether you wanted to also know the specific state of each underlying filter that the IMM maintains under the tracker. That is possible, albeit requires a little bit of code as you can see below.

% You create a TOMHT tracker object with initekfimm
tracker = trackerTOMHT('FilterInitializationFcn', @initekfimm);

% You run it at least once on your data, myDetections, and update the
% tracker to time. Use three outputs to get all the tracks: allTracks.
[confirmedTracks, tentativeTracks, allTracks] =  tracker(myDetections, time);

% In a loop, loop over allTracks
for trackIndex = 1:numel(allTracks)
    % Get the trackID to use with the method getTrackFilterProperties
    trackID = allTracks(trackIndex).TrackID;
    
    % From the IMM filter, get the TrackingFilters property, which contains
    % all the filters. The method returns a cell array of results.
    filtersCell = getTrackFilterProperties(tracker, trackID, 'TrackingFilters');
    
    % The first element in the cell is the TrackingFilters property. Please
    % be careful with those, because they are handle objects. Any
    % change you made to them will impact the state of the filter.
    filters = filtersCell{1};
    
    % You can ispect the state of each filter internally.
    for filterIndex = 1:numel(filters)
        disp("TrackID: " + trackID + " Filter # " + filterIndex + " state is:");
        disp(filters{filterIndex}.State);
    end
end

[Joost]

Thanks you very much for the thorough answer, everything is clear to me now. Kudos on the architecture of the whole toolset, especially regarding the flexibilty in the choice of filters.

[Joost]

I have one more question. I see in the algorithm flow of both the TOMHT and the JPDA tracker that the prediction+correction is done at the end of the flow. This means that the association of new detections are based on the location of the track from the previous time step. Is this true?

In other words, if a ship is at (0, 0) at t=0, then moves to (40, 0) at t=1. Then at t=1, detections come in. Will the detections close to (0,0) be prefered to those close to (40, 0)? (I understand that you do not know where the ship will be at t=1, but a constant velocity preditiction for example might be close to (40,0).

Kind regards,

Joost

[Prashant Arora]

Hi Joost,

Short answer is No, its not true. The tracker will predict the track to detection time for association.

Details:

To associate detections to tracks, the tracker calculates an association likelihood in form of a "cost matrix" or "likelihood matrix" between incoming detections and existing tracks.

The association likelihood will be calculated between every pair of incoming detections and tracks. For this, the tracker will first predict the track to the detection time and then calculate the likelihood of association. The tracker is smart enough to not affect the state of any track due to this prediction. Once, an association matrix is computed for all possible associations, the tracker will decide the probability of each association hypothesis/event based on parameters like clutter density, detection probability etc. Once the tracker knows which detections "should be" assigned to a track, it will do a final predict/correct using the associated detections.

To answer your question about a ship at (0,0) at t = 0 and new detections at t = 1. Let's assume that the detection at t = 0 initiated the track. At this initialization, we create a new track and its state estimator is defined using the FilterInitializationFcn. The FilterInitailizationFcn specifies the initial state of the track as well as initial state covariance as a function of detection by using the "inverse sensor model". All the pre-built FilterInitializationFcn will initialize a track with 0 velocity, if the detection didn't contain any velocity information. To accomodate for this guess of 0 velocity, we specify a default velocity estimate variance to a large number (chosen as 100 arbitrarily by pre-built FilterInitializationFcn). Because initial track state has 0 velocity, the prediction at time t = 1, will not move the "mean" state of the track, but will produce an increased uncertainty in position due to the specified state covariance.

If you have prior information about the "expected" velocity of your ship, you can modify this initial guess by modifying any of the FilterInitalizationFcn and creating your own function. An example to accomodate this "simple ship example" is:

function filter = myCustomInitFcn(detection)
    filter = initcvekf(detection); % Default
    filter.State(2) = 40; % I know expected velocity of my ship in X direction
    filter.State(2,2) = 10; % I know the variance in my expected velocity
end

Any level of customization can be done in a FilterInitializationFcn depending on your particular application. You can then use this function with trackerGNN, trackerJPDA or trackerTOMHT as the FilterInitializationFcn.

tracker.FilterInitializationFcn = 'myCustomInitFcn';

Hope this helps.

Prashant

[Joost]

Thank you Prashant,

this definitily helped a lot. I have some more questions about this prediction for the TOMHT tracker: ("For this, the tracker will first predict the track to the detection time and then calculate the likelihood of association.").

1. Does it use for this prediction the same filter as in step 11 of the algorithm logic flow? Is this prediction also corrected by the filter? I assume that it is, and the outcome of this is used to score the branch?
2. And is this prediction done from the state & covariance matrix resulting from the previous prediction of this branch? (Or is it done from the state & covariance matrix of the track?)
3. And In step 11, how does it combine the final scores of each branch to a new state & covariance matrix of the track? I assume, that the previous predicted and corrected states of each branch that were used for the branch scoring, are predicted even further to the input time, and the weighted average of these predictions is taken. But if you use an IMM filter, this seems difficult because you are now predicting ahead of the measurement, so there is no way to weight the different filters used by the IMM.

Kind Regards,

Joost

[Prashant Arora]

Hi Joost,

In general, data association for TOMHT is carried out at the branch level. Every branch carries its own filter (with the same filter type and initial state as specified by FilterInitializationFcn).

1. There are essentially two types of "prediction". The first one is to calculate association likelihood with every detection that can into the step. This prediction is NOT followed by a correction and the filter is brought back to its original state. The second type of prediction is when the branch must be updated with a particular detection. In that case, we perform a prediction + correction AND yes, its outcome is used to update the branch score. For every prediction, correction or likelihood, distance calculation, we use the same filter as initialized by FilterInitializationFcn. The tracker essentially calls these methods on the filter. For example, if your FilterInitializationFcn is outputting an IMM filter, every branch is estimated using an IMM filter.
2. Yes, every branch maintains its own filter and estimate, which is used for prediction.
3. This is correct. Every branch is predicted to the input time and the "output" tracks is the weighted mean (Gaussian-mixture reduction) of individual branch estimates. If every branch is using an IMM filter, the model probabilities of motion models per barnch will also be "predicted" to input time. The IMM filter standard model probabilities update is split into two parts - prediction and correction. This is provided in the reference literature [1] for IMM filter. During prediction, we predict them using prior information provided via "transition probabilities". This essentially captures the effect of "dT" on the probabilities in the absence of measurement. If measurement is indeed available, we can simply "correct" these predicted probabilities using the measurement likelihood.

Interacting multiple model (IMM) filter for object tracking - MATLAB (mathworks.com)

Hope this helps. All the algorithms are implemented using MATLAB code. You should be able to validate most of what I said by opening the respective files.

Thanks,

Prashant