Tracking a clay target using foreground detection using Gaussian mixture models inconsistently identifying the target.

I am working on a project to use stereo vision to track a shooters aim and a clay target in flight and try and calculate the distance the shooter misses by. I have colored markers on the gun to assist with the aim but I am having trouble tracking the clay target.

I am using some code i have adapted from one of the online examples, which so far identifies the target in only a few out of the 10 frames in my video. Any help that you can give is greatly appreciated.

The code that i am currently using is:

% defines min/max blob sizes
minsize = 8;
maxsize = 100;

videoSource = vision.VideoFileReader('L_Trial2.avi','VideoOutputDataType','uint8');

detector = vision.ForegroundDetector('NumTrainingFrames', 5,...
    'InitialVariance', 200, 'NumGaussians', 8, 'MinimumBackgroundRatio', 0.1);

blobbbox = vision.BlobAnalysis(...
       'CentroidOutputPort', false, 'AreaOutputPort', false, ...
       'BoundingBoxOutputPort', true, ...
       'MinimumBlobAreaSource', 'Property', 'MinimumBlobArea', minsize,...
       'MaximumBlobAreaSource', 'Property', 'MaximumBlobArea',maxsize);

   shapeInserter = vision.ShapeInserter('BorderColor','White');

   videoPlayer = vision.VideoPlayer();
  for i = 1:10
       frame  = step(videoSource);
       fgMask = step(detector, frame);
       bbox = step(blobbbox, fgMask);
       out = step(shapeInserter, frame, bbox);
       step(videoPlayer, out);
       pause(0.5)
  end

release(videoPlayer);
release(videoSource);

Links to this .m file, and the video i am using are:

https://www.dropbox.com/s/w3ysdsgjuj9sqa9/GetTargetLocCutdown.m?dl=0

https://www.dropbox.com/s/0vshthqtha5i6by/L_Trial2.avi?dl=0

[Cedric]

Download link: https://www.dropbox.com/s/0vshthqtha5i6by/L_Trial2.avi?dl=1

updated: https://www.dropbox.com/s/9iy0m5n20t9k39v/GOPR1060.MP4?dl=1

[Josh Anderson]

Two updated links with more video frames:

Updated Video https://www.dropbox.com/s/9iy0m5n20t9k39v/GOPR1060.MP4?dl=1

Updated .m file https://www.dropbox.com/s/n14obw97hxgmcbd/GetTargetLocCutdownUpdated.m?dl=1

Answer 1

[Josh Anderson]

I have tried a few possible solutions since posting this question and found one that meets my needs. If anyone is interested, the code that I have come up with is:

TrialNum = 2;
numframes = 10;
folderpath = 'E:\Gopro\20150820 - dynamic trials\ToVideo\';

STN = int2str(TrialNum);
STNL = strcat('L_Trial',STN,'.avi');
% STNR = strcat('R_Trial',STN,'.avi');

minsize = 12;
maxsize = 100;

videoSource = vision.VideoFileReader(fullfile(folderpath,STNL),'VideoOutputDataType','uint8');

blobarea = vision.BlobAnalysis(...
       'CentroidOutputPort', false, 'AreaOutputPort', true, ...
       'BoundingBoxOutputPort', false, ...
       'MinimumBlobAreaSource', 'Property', 'MinimumBlobArea', minsize,...
       'MaximumBlobAreaSource', 'Property', 'MaximumBlobArea',maxsize);

blobcent = vision.BlobAnalysis(...
       'CentroidOutputPort', true, 'AreaOutputPort', false, ...
       'BoundingBoxOutputPort', false, ...
       'MinimumBlobAreaSource', 'Property', 'MinimumBlobArea', minsize,...
       'MaximumBlobAreaSource', 'Property', 'MaximumBlobArea',maxsize);

% Read first frame to use as initial background image
frame  = step(videoSource);
targarea2 = frame(300:600,1000:1800);

% loop and save target centroid location and area for each frame
for i = 1:numframes-1

    frame  = step(videoSource);
    targarea1 = frame(300:600,1000:1800);

    output = targarea2-targarea1;  

    targarea2 = targarea1;

    output = output > 15;  

    cent = step(blobcent, output);
    cent(1) = cent(1)+1000;
    cent(2) = cent(2)+300;
    area = step(blobarea, output);  
    trackerarray(i,:)=[area, cent];
  end    

release(videoSource);

This takes identifies the target in each frame except the first one (which i can live with), and returns the target size and location.

This doesn't use Gaussian mixture modelling as the question title suggests but it works for me.

Josh Anderson

Answer 2

[Dima Lisin]

Hi Josh,

There are several issues here. One is that the target is very small. I had to watch the video a few times before I even noticed it. It would help if you could make the target look bigger in the video. You could move the camera closer, use a zoom lens, or a use higher resolution camera.

The other issue is that you only have 10 frames. vision.ForegroundDetector learns the background over time. If you start recording some time before the first target is launched, it will give the foreground detector time to learn what the background pixel values are, and it will be better able to distinguish the target from noise.

[Josh Anderson]

Thanks for your response. I updated my program to read directly from the original .mp4 file, which has approximately 140 frames before the original video I posted begins.

I increased the training frames to 140, and changed no other parameters. Now it doesn't detect the target in any of the frames where its flying.

I have added links to the new files as comments in the original question.

I agreed that ideally the target would be visibly larger, the issue I face is that I am trying to film at 60fps(see updated Video) to reduce syncronisation errors between my cameras, currently the resolution is 1920x1080 and the target is ~12m away, for it to be a realistic shooting situation I cant really adjust the layout to move the target closer.

Would you know of a different Matlab object/routine that would identify the target more reliably in this situation?