Segmenting Matlab's Car image - Discussion & Conceptual

Hi,

I am writing a lot of detail in the question below about how I tried to think about an approach so I can get some guidance on how to correctly build an approach to segmenting images.

Disclaimer: I am new to Image Processing. Anything I know about it is simply from reading examples, referring documentation, and then going to read the concept behind it for the past several days. I did the BlobsDemo from ImageAnalyst to get a starting idea on Matlab's functionality as well.

In actual application, I think segmenting won't be a challenging task for me as I will have a constant background and changing objects, so I am sure I could use imabsdiff().

However, just to sharpen concepts I attempted to segment out the car in Matlab's 'car2.jpg' (attached for convenience) and failed at my attempts in segmenting the car. Noting also that it should be done to some extent in an automated fashion (i.e. the code isn't designed specifically to fit this image, but many images as such - although my attempts below are the result of starting at a base and adding on based on responses I saw in the image)

I tried experimenting with two ways. First was the accepted answer on this question: https://www.mathworks.com/matlabcentral/answers/346309-help-me-please-how-to-crop-image-remove-background-of-currency

I understand the code provided in that link is for a simply image with a more distinguishable background, but just to use as a starting point. So far, I think of saturation in hsv as values closer to 255 on either the R,G,B channel; so obviously there are several spots in the background that I have a problem with the saturation. In the link, the mask is created from a Saturation Image that is comprised of regions greater than 0.1 Saturation Value. In my code, I tried 0.35, and got most of the car, but still was getting a lot of the background. I also first assumed that since the sky is 'bright' in this picture with the sun shining, I added another mask to the first masked image to take out higher saturation values to see if I could get rid of the bright part of the background (I tried <0.9 and various values in that region), and it didn't do anything. Thus, conceptual error

Another code I tried was the below; the comments on the code hint at what I was trying to study about the response:

[filename,filepath] = uigetfile('*.*','Select'); %select the car2 image
car_rgb = imread(fullfile(filepath,filename));
car_rgb = imresize(car_rgb,0.5);
thresholdValues = [.2,.4,.6,.8]; %initially this was to loop through various values and see how the output behaves... not very automated I know
car_unfiltered = rgb2gray(car_rgb); %continuing from the above - hopefully learning something from the responses would help build a more automated approach
sz = size(thresholdValues,2);
for i = 1:sz
    %since I kept losing the top right half of the car to the background in
    %other attempts, I assume that pixels there are too far apart to be
    %connected - so try filtering to bring them closer - blur the image
    
    %try with median filtering
    car = medfilt2(car_unfiltered);
    thresholded = imbinarize(car,thresholdValues(i));
    subplot(4,4,i)
    imshow(thresholded)
    thresholded = imfill(thresholded,'holes');
    thresholded = bwareaopen(thresholded,50); %can experiment with moving this command before the previous one
    blob = bwareafilt(thresholded,1); %hopefully picked up most of the car
    L = bwlabel(blob);
    B = labeloverlay(car_rgb,L);
    subplot(4,4,i+4);
    imshow(B);
    
    %try with gaussian filtering
    car = imgaussfilt(car_unfiltered);
    thresholded = imbinarize(car,thresholdValues(i));
    subplot(4,4,i+8)
    imshow(thresholded)
    thresholded = imfill(thresholded,'holes');
    thresholded = bwareaopen(thresholded,50);
    blob = bwareafilt(thresholded,1);
    L = bwlabel(blob);
    B = labeloverlay(car_rgb,L);
    subplot(4,4,i+12);
    imshow(B);
end

After that I tried the below

%since getting most of the car and the top right of the image in my blob in
%the lower threshold.. and only getting the top right in the high
%threshold.. trying to take a difference. If this were to work it could lay
%ground to a more automated approach with some conditional checking on
%images

car = medfilt2(car_unfiltered);
thresholded = imbinarize(car,.2);
thresholded = imfill(thresholded,'holes');
thresholded = bwareaopen(thresholded,50); %can experiment with moving this command before the previous one
blob = bwareafilt(thresholded,1);
L1 = bwlabel(blob);

car = medfilt2(car_unfiltered);
thresholded = imbinarize(car,.6);
thresholded = imfill(thresholded,'holes');
thresholded = bwareaopen(thresholded,50); %can experiment with moving this command before the previous one
blob = bwareafilt(thresholded,1);
L2 = bwlabel(blob);

L3 = L1 - L2;

B = labeloverlay(car_rgb,L3);
subplot(1,1,1);

imshow(B);

The results are attached in the file 'labeloverlay2'