How to apply attention mechanism to object detection in RGB images

Hello, as this is my first time using MATLAB for research in deep learning, I am not very proficient yet. Can someone give me an example of how to apply attention mechanisms such as attention layers and self-attention layers to object detection in RGB images? Thank you very much.

Answer 1

[Image Analyst]

I don't know what an attention mechanism is.

Try these links on abandoned object detection:

https://www.mathworks.com/search.html?c%5B%5D=entire_site&q=abandoned&page=1

https://www.mathworks.com/help/vision/ug/abandoned-object-detection.html?s_tid=srchtitle_site_search_1_abandoned

Or maybe you're looking for "saliency":

https://www.mathworks.com/support/search.html?q=saliency&page=1

[Tian,HCong]

Thank you for your prompt reply. I apologize for not describing my problem clearly.

I will use the following link as an example to restate my question. As shown in the link, SelfAttentionLayer is used to process 28x28 grayscale images. How can I modify the program if I replace the graphics with RGB images? Can you give me a simple example or give me some guidance?

Example of using Self attention layer in MATLAB R2023A - MATLAB Answers - MATLAB Central (mathworks.cn)

Thank you very much.

[Image Analyst]

Looks like you'll need to replace your images with gray scale images. You can either use rgb2gray or you can simply take one of the color channels, whichever one has the best contrast and least noise. Save them into a different folder. Here is a little script to convert all your images to 28x28 grayscale and save them to a different folder:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format short g;
format compact;

% Specify the folder where the files live.  CHANGE THESE LINES!
inputRGBFolder = 'C:\Users\yourUserName\Documents\My Pictures';
outputGrayscaleFolder = 'C:\Users\yourUserName\Documents\My Pictures\GrayScale';

% Check to make sure that input folder actually exists.  Warn user if it doesn't.
if ~isfolder(inputRGBFolder)
	errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', inputRGBFolder);
	uiwait(warndlg(errorMessage));
	inputRGBFolder = uigetdir(); % Ask for a new one.
	if inputRGBFolder == 0
		% User clicked Cancel
		return;
	end
else
	% Check to make sure that output folder actually exists.  
	% Create it if it doesn't as a subfolder of the input folder.
	if ~isfolder(outputGrayscaleFolder)
		mkdir(outputGrayscaleFolder);
	end
end

% Get a list of all files in the input folder with the desired file name pattern.
filePattern = fullfile(inputRGBFolder, '*.png'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1 : length(theFiles)
	% Get this filename.
	baseFileName = theFiles(k).name;
	fullFileName = fullfile(theFiles(k).folder, baseFileName);
	fprintf(1, 'Now reading %s\n', fullFileName);
	% Read it in as an image array with imread();
	imageArray = imread(fullFileName);
    
	% Display it.
	subplot(2, 1, 1);
	imshow(imageArray);  % Display image.
	axis('on', 'image')
	caption = sprintf('Original Image : %s', baseFileName);
	title(caption);
    
	% See if it's RGB.
	[rows, columns, numberOfColorChannels] = size(imageArray);
	if numberOfColorChannels >= 3
		% Convert to grayscale
		imageArray = rgb2gray(imageArray);
		% Or alternatively (a little faster):
		%imageArray = imageArray(:, :, 2); % Take green channel.
    end
    
	% Now resize it to 28 x 28 (it will be almost unrecognizable since that is so very tiny).
	imageArray = imresize(imageArray, [28, 28]);
    
	subplot(2, 1, 2);
	imshow(imageArray);  % Display image.
	title('Grayscale Image')
	axis('on', 'image')
	drawnow; % Force display to update immediately.
    
	% Save grayscale version to output folder
	outputFullFileName = fullfile(outputGrayscaleFolder, baseFileName);
	imwrite(imageArray, outputFullFileName);
end