Plot a single pixel's (or pixel region's) intensity over frame number of a monochrome video

Hello,

I have been trying to simply plot one pixel's intensity value throughout a video for a while now and have not found anything that works.

I have tried using "Image Analyst's" demo that does almost this, and cannot get passed the error "MAP requires a 3xm array"

I'd imagine this shouldn't take too much code to get to work.

[Rik]

Did you already solve this? If not, please link to the code you used. It would also be helpful to have an example of your video file.

[Trevor Voss]

Yes! It was pretty simple...

for n=1:size(Movie,2) PixelIntensity(n)=Movie(n).cdata(49,290); end

I just didn't get the ".cdata" part.

[Image Analyst]

As you can see from my demo in my answer below, I just did:

% Calculate the mean R, G, and B levels.
meanRedLevels(frame) = mean(mean(thisFrame(:, :, 1)));
meanGreenLevels(frame) = mean(mean(thisFrame(:, :, 2)));
meanBlueLevels(frame) = mean(mean(thisFrame(:, :, 3)));

Did you try that one? I'm not sure what demo you originally had but if it's the one I think it was, the one I attached below is simplified.

[laurent jalabert]

is it possible to use

[x,y]=ginput(1) 

in order to define 1 fixed point in the video, and get the RGB levels at this fixed point (x,y) ?

RedLevels(frame) = thisFrame(x, y, 1);
GreenLevels(frame) = thisFrame(x, y, 2);
BlueLevels(frame) = thisFrame(x, y, 3);

The global idea is to plot the time dependence of RGB pixels at a fixed point defined by the user on the video.

[DGM]

You can, though it may be worthwhile to sample a region locally around the selected point. If you're only sampling a single pixel over time, noise may be significant (depending on the source and conditions).

This is a chopped-down version of IA's demo below that's been modified to do both.

sampleradius = 4; % set to zero for no averaging

% read the file
videoObject = VideoReader('rhinos.avi');

% Determine how many frames there are.
numberOfFrames = videoObject.NumFrames;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;

% preview frame 1 and let the user pick a location
imshow(read(videoObject,1));

%[x y] = ginput(1); % this can't be done in the forum editor (can't interact)
x = 100; y = 100; % so i'm just inserting dummy values

% expand coordinates to a rectangular range around the point
x = x + (-sampleradius:sampleradius);
y = y + (-sampleradius:sampleradius);

% constrain coordinates to image area
x = min(max(round(x),1),vidWidth);
y = min(max(round(y),1),vidHeight);

% preallocate
meanLevels = zeros(numberOfFrames,1,4);

% Loop through the movie, processing frames
for frame = 1 : numberOfFrames
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, frame);
    
    % Calculate the mean R, G, and B levels.
    meanLevels(frame,1,1:3) = mean(thisFrame(y,x,:),[1 2])/255;
end

% add gray level channel
meanLevels(:,:,4) = rgb2gray(meanLevels(:,:,1:3));

% plot everything
plot(meanLevels(:,:,1),'r'); hold on
plot(meanLevels(:,:,2),'g')
plot(meanLevels(:,:,3),'b')
plot(meanLevels(:,:,4),'k')

[Image Analyst]

@laurent jalabert one important thing to notice between your code and @DGM's comment after it (click the link to show all comments) is that you made the common beginner mistake of doing thisFrame(x, y, 1) while DGM did thisFrame(y, x, 1). Note that matrices are indexed (row, column), in other words (y, x), NOT (x, y).

You can use drawpoint to have the user pick a point on your image to monitor.

help drawpoint

 drawpoint Create draggable point ROI
     H = drawpoint begins interactive placement of a point region of
     interest (ROI) on the current axes. The function returns H, a handle to
     an images.roi.Point object. You can modify an ROI interactively using
     the mouse. The ROI also supports a context menu that controls
     aspects of its appearance and behavior.
 
     H = drawpoint(AX,____) creates the ROI on the axes specified by AX
     instead of the current axes (gca)
 
     H = drawpoint(____, Name, Value) modifies the appearance of the ROI
     using one or more name-value pairs.
 
     Parameters include:
 
     'Color'           ROI color, specified as a MATLAB ColorSpec. The
                       intensities must be in the range [0,1].
 
     'ContextMenu'     Context menu, specified as a ContextMenu object. Use
                       this property to display a custom context menu when
                       you right-click on the ROI. Create the context menu
                       using the uicontextmenu function.
 
     'Deletable'       ROI can be interactively deleted via a context menu,
                       specified as a logical scalar. When true (default),
                       you can delete the ROI via the context menu. To
                       disable this context menu item, set this property to
                       false. Even when set to false, you can still delete
                       the ROI by calling the delete function specifying the
                       handle to the ROI, delete(H).
 
     'DrawingArea'     Area of the axes in which you can interactively place
                       the ROI, specified as one of these values:
 
                       'auto'      - The drawing area is a superset of the 
                                     current axes limits and a bounding box
                                     that surrounds the ROI (default).
                       'unlimited' - The drawing area has no boundary and
                                     ROIs can be drawn or dragged to
                                     extend beyond the axes limits.
                       [x,y,w,h]   - The drawing area is restricted to an
                                     area beginning at (x,y), with
                                     width w and height h.
 
     'HandleVisibility'   Visibility of the ROI handle in the Children
                          property of the parent, specified as one of these
                          values:
                          'on'      - Object handle is always visible
                                      (default).
                          'off'     - Object handle is never visible.
                          'callback'- Object handle is visible from within
                                      callbacks or functions invoked by
                                      callbacks, but not from within
                                      functions invoked from the command
                                      line.
 
     'InteractionsAllowed' Interactivity of the ROI, specified as one of
                           these values:
                           'all'      - ROI is fully interactable (default).
                           'none'     - ROI is not interactable and no drag
                                        points are visible.
 
     'Label'           ROI label, specified as a character vector or string.
                       When this property is empty, no label is
                       displayed (default).
 
     'LabelAlpha'      Transparency of the text background, specified as a 
                       scalar value in the range [0 1]. When set to 1, the
                       text background is fully opaque. When set to 0, the
                       text background is completely transparent. Default
                       value is 1.
 
     'LabelTextColor'  Label text color, specified as a MATLAB ColorSpec. 
                       The intensities must be in the range [0,1].
 
 
     'LabelVisible'    Visibility of the label, specified as one of these 
                       values:
                       'on'      - Label is visible when the ROI is visible
                                   and the Label property is nonempty 
                                   (default).
                       'hover'   - Label is visible only when the mouse is
                                   hovering over the ROI.
                       'off'     - Label is not visible.
 
     'LineWidth'       Line width, specified as a positive value in points.
                       The default value is three times the number of points
                       per screen pixel.
 
     'MarkerSize'      Marker size, specified as a positive value in 
                       points. The default value is eight times the number 
                       of points per screen pixel.
 
 
     'Parent'          ROI parent, specified as an axes object.
 
     'Position'        Position of the point, specified as a 1-by-2 array
                       of the form [x y] where x and y are coordinates of
                       the point ROI. This property updates automatically
                       when you draw or move the ROI.
 
     'Selected'        Selection state of the ROI, specified as true or
                       false. To set this property to true interactively,
                       click the ROI. To clear the selection of the ROI,
                       and set this property to false, ctrl-click the ROI.
 
     'SelectedColor'   Color of the ROI when the Selected property is true,
                       specified as a MATLAB ColorSpec. The intensities must
                       be in the range [0,1]. If you specify the value
                       'none', the Color property specifies the ROI color,
                       irrespective of the value of the Selected property.
 
     'StripeColor'     Color of the ROI stripe, specified as a MATLAB
                       ColorSpec. By default, the edge of an ROI is solid
                       colored. If you specify a StripeColor, the ROI edge
                       is striped, using a combination of the Color value
                       and this value. The intensities must be in the range
                       [0,1].
 
     'Tag'             Tag to associate with the ROI, specified as a
                       character vector or string.
 
     'UserData'        Data to associate with the ROI, specified as any
                       MATLAB data, for example, a scalar, vector,
                       matrix, cell array, string, character array, table,
                       or structure. MATLAB does not use this data.
 
     'Visible'         ROI visibility, specified as one of these values:
 
                       'on'  - Display the ROI (default).
                       'off' - Hide the ROI without deleting it. You still
                               can access the properties of an invisible ROI.
 
 
     Example 1
     ---------
 
     % Display an image
     figure;
     imshow(imread('baby.jpg'));
 
     % Begin interactive placement of a point
     h = drawpoint();
 
     % Add a tag to the ROI
     h.Tag = 'My Point';
 
 
     Example 2
     ---------
 
     % Display an image
     figure;
     imshow(imread('baby.jpg'));
 
     % Programmatically place ROI
     h = drawpoint('Position',[100,100]);
 
 
     See also: images.roi.Point, drawcircle, drawellipse, drawfreehand,
     drawline, drawpolygon, drawpolyline, drawrectangle, drawassisted,
     drawcuboid

    Documentation for drawpoint
       doc drawpoint

[laurent jalabert]

yes thank you, my mistake. I would say that x=y=100 does not change the philosophy of my question. I will use drawpoint function.

[DGM]

The reason for the static values is simply that interactive graphics tools (ginput(), getpts(), drawpoint(), etc) can't be demonstrated on the forum. The dummy values are to be removed and the ginput() line uncommented.

Unless the goal is a mask-based workflow, there really isn't any big advantage to using ROI objects. It offers extra functionality, but I don't see the point in managing the created object if all that's needed are coordinates.

ginput() Get one or more points, returning x,y coordinates and optional keypress info. Part of base MATLAB.

getpts() Similar to ginput(), but controls are slightly different. Requires Image Processing Toolbox.

drawpoint() Full object-oriented tool for interactive generation of point ROI objects. Requires R2018b+ and IPT

Answer 1

[Image Analyst]

The attached demo works just fine and there is no map issue. If you still have a problem, attach your movie in a .zip file.

Answer 2

[laurent jalabert]

I would like to improve this program, because when I create about 20 points to be monitored, the loop duration can reach 50sec per frame (Macbook pro i7, 16Gb DRAM, 1To SSD, Matlab 2021a with Image processing toolbox), and I have 4500 frames in my video. So it takes forever. Therefore I introduced step_frame to analyse a part of the main video, that is 450 frames. Even that, the loop is not optimized.

subplot (221) is showing the actual frame, I want to overlay the array of points selected by the user. Same on subplot(223) and subplot(224). The subplot(222) is a plot of all the selected array of blue intensity data.

This blue intensity is generated from im2gray function.

This program is indeed inspired from the movie_plot_intensity.m, but I adapted to my concern.

Any help for improving the computing time will be welcome.

%% Program that is doing :
% 1- play a video
% 2- place Npts points on this video
% 3- analyse the averaged Gray, R, G, B
% 4- add the BLUE intensity of the Npts points and associate a color
% 5- save as jpeg the figure containing the frames and plot
% 6- export the plot data to excel file.

% the problem of this program is mainly that it takes a lot of time when Npts =10 or 20 ! 
% how to improve it ? 


clc;    % Clear the command window.
close all;  % Close all figures
imtool close all;  % Close all imtool figures.
clear;  % Erase all existing variables.
workspace;  % Make sure the workspace panel is showing.


% declaration of constants
%Npts =20;
start_frame = 1; % start at frame 1
step_frame = 10; % display every 10 frames
fontSize = 16;

cd('yourfolder')

folder = fileparts(which('yourvideo.mp4'));

movieFullFileName = fullfile(folder, 'yourvideo.mp4');

if ~exist(movieFullFileName, 'file')
    strErrorMessage = sprintf('File not found:\n%s\nYou can choose a new one, or cancel', movieFullFileName);
    response = questdlg(strErrorMessage, 'File not found', 'OK - choose a new movie.', 'Cancel', 'OK - choose a new movie.');
    if strcmpi(response, 'OK - choose a new movie.')
        [baseFileName, folderName, FilterIndex] = uigetfile('*.mp4');
        if ~isequal(baseFileName, 0)
            movieFullFileName = fullfile(folderName, baseFileName);
        else
            return;
        end
    else
        return;
    end
end

videoObject = VideoReader(movieFullFileName);

numberOfFrames = videoObject.NumberOfFrames;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;

numberOfFramesWritten = 0;
[folder, baseFileName, extentions] = fileparts(movieFullFileName);

% open the video reader and run the video. 
vid=implay(movieFullFileName);
%CurrentFrameNumber = vid.DataSource.Controls.CurrentFrame

% place points onto the video frame and get coordinates array [x,y]
[x,y] = getpts(vid.Visual.Axes); 
X=x; Y=y;

% the total number of points that were created is Npts
Npts = length(X);
% define 1 color for each points
couleur = parula(Npts);

% this is to discretize 1 line into Npts points (not used here)
%[x,y] = getline(vid.Visual.Axes);
%X=linspace(x(1),x(2),Npts); X=round(X);
%Y=linspace(y(1),y(2),Npts); Y=round(Y);
%hpt = drawline('Position',[x(1),y(1);x(2),y(2)],'StripeColor','white');
%hpt = drawpoint('Position',[x;y]);

% create the legend text relative to the points label 1,2, ... Npts. 
% in this program, the Gray, R, G, B averaged values are also displayed.
for k = 1:Npts
    txt = ['Gray','R','G','B',num2str(1:k)];
end

% for variable names in table (to export to excel)
% when exporting a table to excel, the variablenames VN have to be created. 
C2 = arrayfun(@(x) sprintf('pt%d', x), 1:Npts, 'UniformOutput', false);
VN = [{'time','Gy','R','G','B'},C2{:}];
VN1 = VN';
variablenames = [string(VN1)];

% the video will start from start_frame number
nb = length(start_frame:step_frame:numberOfFrames);

% xx is the time array. 
xx = (1./videoObject.FrameRate).*(start_frame:step_frame:numberOfFrames-start_frame+1);

% initialize the arrays
meanGrayLevels = zeros(nb, 1);
meanRedLevels = zeros(nb, 1);
meanGreenLevels = zeros(nb, 1);
meanBlueLevels = zeros(nb, 1);
SPBLUE = zeros(nb, Npts);

%% step 2 - record the figure sub 4 subplots and make a video
analyse_day= datestr(now,'CP_yyyymmdd_hhMMss'); % data treatment day

% output_video_name = strcat(analyse_day,'_output.mov');
% v = VideoWriter(output_video_name,'Archival');
% v.VideoCompressionMethod
% open(v);

% the background is the first frame (start_frame)

background2 = read(videoObject, start_frame);

%% Main loop to read each frane and compute pixel intensities
hFIG = figure;
set(hFIG, 'units','normalized','outerposition',[0 0 1 1]);

kk=0;
for thisframe = start_frame : step_frame : numberOfFrames
    tic
    kk = kk+1;
    
    set(0,'CurrentFigure',hFIG);
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, thisframe);
    
    subplot(2, 2, 1);
    image(thisFrame);
    caption = sprintf('Frame %4d of %d.', thisframe, numberOfFrames);
    title(caption, 'FontSize', fontSize);
    
    % if I place the labels on this frame, the program is taking more time
    %     if thisframe == start_frame
    %         for jj=1:Npts
    %             hpt(jj) = drawpoint('Position',[X(jj),Y(jj)],'Color',couleur(jj,:));
    %             hpt(jj).Label = num2str(jj);
    %         end
    %     end
    
    % Calculate the mean gray level.
    grayImage = im2gray(thisFrame);
    meanGrayLevels(kk) = mean(grayImage(:));
    
    % Npts Blue pixel intensity (:,:,3)
    for jj=1:Npts
        a = round(Y(jj)); b=round(X(jj));
        spBlue = im2gray(thisFrame(a, b, 3));
        SPBLUE(kk,jj) = spBlue;
    end
    
    % Calculate the averaged R, G, and B levels.
    meanRedLevels(kk) = mean(mean(thisFrame(:, :, 1)));
    meanGreenLevels(kk) = mean(mean(thisFrame(:, :, 2)));
    meanBlueLevels(kk) = mean(mean(thisFrame(:, :, 3)));
    
    % Plot the mean gray levels.
    subplot(2, 2, 2);
    hold off;
    plot(xx,(meanGrayLevels), 'k-', 'LineWidth', 2,'DisplayName','Gray');hold on;
    plot(xx,(meanRedLevels),  'r-', 'LineWidth', 2,'DisplayName','R');hold on;
    plot(xx,(meanGreenLevels),'g-', 'LineWidth', 2,'DisplayName','G');hold on;
    plot(xx,(meanBlueLevels), 'b-', 'LineWidth', 2,'DisplayName','B');hold on;
    
    for jj=1:Npts
        plot(xx,((SPBLUE(:,jj))),'Color',couleur(jj,:),'LineWidth', 1,'DisplayName',num2str(jj));hold on;
    end
    
    legend('Location','NorthEastOutside','NumColumns',1);
    grid on;
    
    % Put title back because plot() erases the existing title.
    title('Mean Pixel Intensity', 'FontSize', fontSize);
    %xlabel('Time (s)');
    %ylabel('Pixel Intensity (a.u)');
    
    numberOfFramesWritten = numberOfFramesWritten + 1
    
    % Display the background image.
    subplot(2, 2, 3);
    image(background2);
    title('background', 'FontSize', fontSize);
    
    % if I place the labels on this frame, the program is taking even more time
    %     if thisframe == start_frame
    %         for jj=1:Npts
    %             hpt(jj) = drawpoint('Position',[X(jj),Y(jj)],'Color',couleur(jj,:));
    %             hpt(jj).Label = num2str(jj);
    %         end
    %     end
    
    
    % plot difference image
    subplot(2, 2, 4);
    
    % Calculate a difference between this frame and the background.
    J = imsubtract(thisFrame,background2);
    
    zb = J(:,:,3);           %get the blue intensity array
    
    I3 = flipdim(zb ,1);   %# vertical flip
    I4 = flipdim(I3,2);    %# horizontal+vertical flip
    I5 = flipdim(I3,1);
    
    myImage = imshow(I5, 'Colormap', jet(256));
    
    axis on;
    colorbar;
    caxis([0, 200]);
    hold on
    
    %     if thisframe == start_frame
    for jj=1:Npts
        plot(X(jj),Y(jj),'+','Color',couleur(jj,:));
        textLabel = sprintf('(%.0f)', jj);
        text(X(jj),Y(jj), textLabel);
        hold on
        %hpt(jj) = drawpoint('Position',[X(jj),Y(jj)],'Color',couleur(jj,:));
        %hpt(jj).Label = num2str(jj);
    end
    %     end
    %drawnow; % Force it to refresh the window
    
    title('Image - Background', 'FontSize', fontSize);
    
    % making a video from the figure is generating some error randomly.
    %F = getframe(hFIG);
    %writeVideo(v,F.cdata);
    
    cd('your output folder')
    savedir = strcat(analyse_day,'_',sprintf('%04d',thisframe),'.jpg');
    saveas(hFIG,savedir);
    toc
end

%close(v);
delete(findall(0))

% array to be saved
data = [xx',meanGrayLevels,meanRedLevels,meanGreenLevels,meanBlueLevels,SPBLUE]
% find any remaining zeros and remove them
IND = find(data(:,2)==0);
data(IND,:)=[];


%% create the plot shown in of subplot(222) to check the data to be exported.
% this is not mandatory and it is done outside of the loop, so it is not
% time consuming.

couleur = parula(Npts);

size_data = size(data,2);

figure('PaperUnits','inches','PaperType','A4','PaperOrientation',...
    'landscape','Color',[1 1 1], 'OuterPosition',[1 1 800 800]);

xx = data(:,1);
plot(xx,data(:,2),'k-', 'LineWidth', 2,'DisplayName','Gray');hold on;
plot(xx,data(:,3),'r-', 'LineWidth', 2,'DisplayName','R');hold on;
plot(xx,data(:,4),'g-', 'LineWidth', 2,'DisplayName','G');hold on;
plot(xx,data(:,5),'b-', 'LineWidth', 2,'DisplayName','B');hold on;

for jj=1:Npts
    plot(xx,((data(:,5+jj))),'Color',couleur(jj,:),'LineWidth', 1,'DisplayName',num2str(jj));hold on;
end

HANDLE = gca;
get( HANDLE );
set( HANDLE, 'Color','w');% [0.8,0.8,0.8] )

legend('Location','NorthEastOutside','NumColumns',1);
grid on;


xlabel('time (s)'); ylabel('Blue Intensity');
set(gca,'FontSize',16');
grid on

for jj = 1:Npts
    xpos = data(end,1);
    ypos = data(end,5+jj);
    labels = num2str(jj) ;
    h = text(xpos, ypos, labels,'FontSize',16)
end

cd(folder);

%%
output_name = strcat(analyse_day,'_',baseFileName,'_','GrRGB_',num2str(Npts),'pts.xlsx');
output_array=array2table(data,'VariableNames',variablenames);
writetable(output_array,output_name);

[Image Analyst]

It may be slowing down because all your images are "stacking up" in your subplots. When you call subplot(), but before you call image() or imshow(), call cla. That will clear the axes so your memory usage won't keep climbing. Or if you want the fastest speed possible, just don't display anything at all inside the loop. Just display things once outside the loop (after it's finished).

[laurent jalabert]

thank you for the advices. Indeed, the aim of displaying everything in the loop is that I record the figure and I later, I gather those images to create a movie, showing nicely what is happening during this video. It is indeed easy to save only 1 figure once, at the end, but this does not show the evolution of some events happening during this video.

[laurent jalabert]

the "cla" in subplot(224) was added, and it is amazingly solving my initial problem. Really thanks a lot !

subplot(2, 2, 4);
cla