Can I use PCA effectively on a greyscale image?

Hello!

I found this code online:

I = double(imread('peppers.png'));
X = reshape(I,size(I,1)*size(I,2),3);
coeff = pca(X);
Itransformed = X*coeff;
Ipc1 = reshape(Itransformed(:,1),size(I,1),size(I,2));
Ipc2 = reshape(Itransformed(:,2),size(I,1),size(I,2));
Ipc3 = reshape(Itransformed(:,3),size(I,1),size(I,2));
figure, imshow(Ipc1,[]);
figure, imshow(Ipc2,[]);
figure, imshow(Ipc3,[]);

provided from another commentary form I was reading and I was wondering if there was any code that performed PCA that did not give the pc as a color channel. I don't want to do PCA on the colors of the image composite I want something else (not sure what else is but something not color), so applying PCA to a gray scale image.

Thanks for you any and all suggestions.

[Image Analyst]

Well with an RGB image, there are 3 independent features - the R, G, and B values. Do you have multiple independent components in your grayscale image? If so, what are they? You must have some reason for thinking you can do this, so . . . what is it? Or better yet, just show your real image and tell us what you want to measure in it and we'll tell you if PCA is the best approach or if there is a better approach. As it is, I don't see what PCA can get you.

[Neo]

Sorry I just saw this post Analyst. And you're absolutely right but what qualifies as being an independent component in a grayscale image? Then I can tell you more clearly. I think its easier to distinguish the components in a RGB since by definition, RGB is what those components are. :P

[Image Analyst]

If you think of a PCA coordinate system as a 3-D rotation of an original system, then I think that might help you. Imagine you have a cigar shape gamut (scatterplot of points in RGB color space). If the cigar didn't align with any of the R, G, or B axes but was at some crazy angle, then you could do PCA on it to get a new rotated system where one PC axis does through the main axis of the cigar, and the two others are perpendicular to that and lie in the small cross-sectional plane of the cigar. But with only 1 axis, how can you think that you can rotate it?

What about my last question where I said "better yet, just show your real image and tell us what you want to measure in it and we'll tell you if PCA is the best approach or if there is a better approach." Can you do that, or are you just trying to get a theoretical intuitive understanding of what PCA is and can do?

[Neo]

Oh I see, but just because an image is not RGB, it doesn't necessarily mean that it has only one axis, it might mean that the image has axes that are not related like in a RGB image. And let me get a picture for you to give an idea of what I am talking about actually, great suggestion. I read your response to quickly. Here ya go:

Answer 1

[Walter Roberson]

Sure. For example,

I = double(imread('cameraman.tif'));
X = reshape(I,[],4);
coeff = pca(X);

This would correlate vertical quarters of the image.

[Neo]

Thank you for your answer! Before I accept, I have a few questions about the image, first how can I resize it to fit the PCA'd image to my screen?

I = double(imread('cameraman.tif'));

X = reshape(I,[],4);

coeff = pca(X);

imshow(X);

Warning: Image is too big to fit on screen; displaying at 3% In imuitools\private\initSize at 71 In imshow at 282

[Walter Roberson]

X = imresize( reshape(I,[],4), 768, 4 );

You might lose a lot of the information this way, which would affect your pca. You could just live with the imshow() warning or you could use imshow() with an InitialMagnification setting, or you could do the pca on the file array but imshow only a reduced array

imshow( imresize(X, 768, []) )

[Neo]

Hey sorry for long response, it took me a minute to digest what you gave me. I used this code (not quite sure why you casted to a double):

I = double(imread('cameraman.tif'));

X = imresize( reshape(I,[],4), 768, 4 );

coeff = pca(X);

And it gave me this error, not sure how to fix it:

Error using imresize>parsePreMethodArgs (line 358)
Invalid input syntax; input argument # 3 is unrecognized

Error in imresize>parseInputs (line 248)
[params.A, params.map, params.scale, params.output_size] = ...

Error in imresize (line 141)
params = parseInputs(varargin{:});

Then I used this code out of curiosity, but it gave me the same error as above:

I = double(imread('cameraman.tif'));

coeff = pca(X); 
imshow( imresize(X, 768, []) );

How can i fix this?

[Walter Roberson]

X = imresize( reshape(I,[],4), [768, 4] );

[Walter Roberson]

Note that Image Analyst's concerns are quite valid. Doing the pca the way shown here would not often be useful; I am just showing that it can be done.

[Neo]

Code and error

I = double(imread('cameraman.tif'));

X = imresize( reshape(I, [],4), [768,4]);

coeff = pca(X);

imshow(X);

Warning: Image is too big to fit on screen; displaying at 67% In imuitools\private\initSize at 71 In imshow at 282

Only a sliver of white appears in the display box. I am a little new at Matlab, I am more fluent in other languages, but the syntax is similar so I am not sure why this isn't working.

[Image Analyst]

What we're saying is just that because you can do PCA on it, say by artificially chopping the image up into spatially separate parts, does not mean that the PCs in that case will mean anything. If you were to take the image you just posted and try to describe in words, what the different PC axes (directions) mean, then what would you say?

[Neo]

I don't know yet. I am trying to get the code to work right now, then I can elaborate. Can you tell me anything in regards to the picture I posted from your first commentary? Also, the question that I asked initially is still incomplete, I'm getting this error for the code on the cameraman image:

Warning: Image is too big to fit on screen; displaying at 67% In imuitools\private\initSize at 71 In imshow at 282

Only a sliver of white appears in the display box. I am a little new at Matlab, I am more fluent in other languages, but the syntax is similar so I am not sure why this isn't working.

But I need to see this image to go further with my analysis and ultimately answer your question. This is just a bug as of now.

[Walter Roberson]

It is a warning due to the small size of your screen. You can eliminate it by using the following, which will further reduce the amount of information available to the pca routine.

X = imresize( reshape(I, [],4), [512,4]);

As I wrote above,

"You could just live with the imshow() warning or you could use imshow() with an InitialMagnification setting, or you could do the pca on the file array but imshow only a reduced array"

A narrow sliver is what you should expect, by the way, since this code is rearranging the image to be 4 pixels wide.

[Image Analyst]

What the reshape does is to rearrange your image into a 4-column array. The leftmost column, column 1, is the left-most quarter of the image. For example, if you're using the cameraman image with 256 columns, then column 1 of X is columns 1-64 of the cameraman image, I, all strung together in one big long column vector. Then column 2 of X is columns 65-128 of I. And column 3 of X is columns 129 - 192 of I. And column 4 of X is columns 193-256 of I. So basically each column of X is a vertical quarter panel of the original image.

So, now that you understand what X in this example is, is there any reason to believe that there is any substantive, deterministic difference between any of the columns for your pattern image, or the cameraman image? Not really. Nothing that I would expect. So if you get 4 PCs from that, then the axes will probably be pretty much random. The PC1 may be somewhat correlated with how the mean brightness changes as you go from one vertical quarter-panel to the next, but who cares? What good is that? None really.

So you really need to figure out what you might expect to get out of doing PCA, rather than just do it and hope, without any basis, that what you get will actually mean something useful.

[Neo]

4 pixels wide because these are the PC values of the camera man image? How can I adjust the magnification setting? I do not see code for it unless you are talking about a setting on the image box itself.

Image Analyst: You're saying PCA on the image would give me a sliver of an image with random patterns from my image?

[Image Analyst]

You could call it up with imtool() if you want to play around with the zoom and scroll. For example:

grayImage = imread('moon.tif');
imtool(grayImage);

Yes, the PCs made from vertical panels of an arbitrary gray scale image would probably be random/nonsense/meaningless.

[Walter Roberson]

http://www.mathworks.com/help/images/ref/imshow.html#input_argument_namevalue_initialmagnification

The 4 has nothing to do with it being cameraman; it is only because in my initial example I used

X = reshape(I,[],4);

"This would correlate vertical quarters of the image"

This was an example showing that pca can be done on a grayscale image. The trick is in doing it meaningfully.

[Walter Roberson]

Your texture-style image no longer appears to be posted ?

[Neo]

What? That's odd. I see it on my browser, hold on let me repost it.

[Neo]

Do you see it now? And I will try the zoom to look at what I get.

[Neo]

So it would be iptgetpref('fit') (default)? Sorry the link provided does not have an explicit example.

[Image Analyst]

Your image is no longer attached. I don't know what happened to it.

Anyway, I'm still waiting for some kind of explanation of WHY you want to do PCA on a gray scale image. Just seeing that texture image you used to have here does not explain why. What do you want to find in that image anyway?

You can ignore the warning. If you want to suppress it completely, run the attached code, or at least the one line in it that refers to that specific warning message.

[Walter Roberson]

imshow(X, 'InitialMagnification', 2/3)

[Neo]

The image is attached, sorry about that, my computer was being odd. Thanks Analyst. I will get back to you tomorrow. Roberson, what does the 2/3 mean?

[Image Analyst]

The 2/3 means to (de)magnify the image by a factor of 2/3, so it will show up on screen with 2/3 as many screen pixels across as the native image resolution. So if the image was 1800 pixels across, it will display it as 1200 pixels across.

[Walter Roberson]

If you are looking for the angle of the texture then you might possibly be able to use radon transform of a thresholded version of the image.

[Neo]

Is there anyway that the "PCA'd" image won't come out looking as a sliver? Like it's principal components still as large as the image was before it was PCA'd?

[Neo]

Analyst, I haven't forgotten your inquiry, an accompanying retort awaits. :p

[Image Analyst]

I could be wrong but I think you'll have as many PCA images as original images and they will have the same size as the original images. So for an RGB image (3 gray scale images), you'll have 3 PCA images. For a database of 1000 gray scale face images, you'll have 1000 "eigenface" images of the same size as the input face images. This is how they build up images of the "average" face. With Walter's example, you had 4 input column vectors and so you'll have 4 output column vectors. If you display one, it will be a sliver. If you want them as 4 images, you'll have to reshape them afterwards, like I do in the attached demo (which I think you've already seen).

[Neo]

Hey Analyst, if one believes that PCA is only meaningful when the image being PCA'd is a RGB image or color image, then how come the PCA of the color image is not the individual color channels?

[Neo]

Thank you for the attached demo. For your first sentence "I think you'll have as many PCA images as original images and they will have the same size as the original images." I believe that is only applicable for color images though, correct? And also, can you identify what reshaping you did in the example? It just appears that the PCAs images are shown. Did they not come out like that?

[Image Analyst]

"PCA is only meaningful when the image being PCA'd is a RGB image" <-- I did not say that! I would not say that because it's not true. PCA can be used on features. The features can be individual R, G, or B color channels, OR they can be individual gray scale images (like a set of faces images), or they can be a feature vector of a bunch of measurements (like mean, circularity, solidity, and mean intensity, or whatever you want).

IF the features are color channels, then the main PC will lie along the major axis of the color gamut, and the two other PCs will lie perpendicular to them. It's like a rotation of coordinate system. See this visualization from the side:

and this visualization of the gamut from the top down:

The first PC would be along the V (vertical axis) and the other two would be radially extending out from there, one going out at 15 degrees and one going out at 65 degrees. So you have a rotated axis system where you could represent any RGB or HSV value in coordinates of that new rotated coordinate system instead of the original RGB coordinate system.

Doing PCA will give you as many PCs as features. If you're doing PCA on a set of 1000 face images, you have 1000 features because each grayscale face image is a feature. So you'd have 1000 PCs. Each PC image is called an eigenface in face recognition parlance. You could then reconstruct virtually any face in the database by taking 10 or 20 eigenface images. The more PCs you include in your reconstruction, the more accurate you're building the face. Of course using all 1000 would rebuild the face exactly to the exact gray level for every pixel.

[Neo]

I think the significance I am looking for is in the PCA of a feature vector in the sense of the example that you used above. "(like mean, circularity, solidity, and mean intensity..." What does HSV mean? I see. I have another question I am about to ask which you may find interesting. To give it "justice" I am starting a different question post, maybe I can give a more distinguished set of answers sides the one I gave to you and Stefan below.

[Neo]

Analyst, how come when doing PCA on colored images you can load in a single image? Does the colored channels count as multiple images i.e. multiple vectors?

[Image Analyst]

Well that's fine if you want to define features. Like Walter suggested, maybe get the orientation with radon(). Or maybe use graycoprops() to get some texture metrics. Or maybe some other measurements like area fraction or whatever. Then if you look at these on a scatterplot and see some pattern, and want to make a model, then PCA may be appropriate.

A colored image counts as three features because we split the color image up into three independent, separate color channel variables before we ever run PCA.

[Neo]

"If you are looking for the angle of the texture then you might possibly be able to use radon transform of a thresholded version of the image." Is what dear Walter suggested, but I don't think I want the angle of the texture, I simply want from that image its principal components. I don't want to use graycoprops because I don't want to normalize the gray-level co-occurrence level in the matrix, I want the texture as it is, unless one of these methods you mention simply models the texture as it is to observe possible patterns, but I don't think they do from what I've read about it. What would these metrics give me that would allow you to think it more appropriate to use PCA, than simply applying PCA on the image directly? And are you saying PCA on the scatterplot or on the image after imposing the comparison metrics?

[Image Analyst]

You can get different PCs depending on what you feed pca(). I don't think you want any old arbitrary PCs. I think you want some that mean something to you. So you can't feed it any old arbitrary data, such as strips of the image, you have to feed pca() data that is also meaningful in advance. Otherwise, it's the well known gigo.

[Neo]

Thank you.

[Image Analyst]

By the way, here is one of the "average faces" of females constructed from eigenfaces. It's often asserted that the average face is the most attractive. However high fashion models that walk the catwalks seldom have faces that look like "average"

[Neo]

Haha, thanks Analyst. But I am more concerned with how I can feed multiple images into the PCA code so that I can get more than one PC from the image. Do you have an average face answer to that? Cheers, Neo Cornel

Answer 2

[mugahid albadawy]

i ve already used the same function for dicom image but it's not working properly

Answer 3

[Stefan Karlsson]

I think the answer by Walter, while technically correct, confounds a VERY simple topic. Being technically correct here amounts to what exactly?

[Neo]

The point of the discussion Stefan. :)

[Image Analyst]

But we're still trying to figure out what you think the PCs should represent.

[Neo]

Exactly or quite similar to the example you gave above. "(like mean, circularity, solidity, and mean intensity, or whatever you want)."

[Stefan Karlsson]

1. A single RGB image, of W x H size. Is there a way to perform PCA in a meaningful way?

answer: yes. the first PC-basis is usually the Brightness component in natural images. Of course, you can create images where this is not true, by forcing all pixel values to have the same brightness you will make the brightness component be the last PC instead of the first.

2. A single gray scale image, of W x H size. Is there any way to do PCA in a meaningful way?

Answer: no.

3. Performing multiple feature extractions, over local regions of a gray-scale image I can gather a "feature image". Can i do PCA on this image in a meaningful wau?

answer: yes. The easist example is if you gather 3 features over each local region. The resulting feature image can then even be visualized as an RGB image. On this image you can do PCA. As example, you can collect the TRIPLET: (mean, median, variance). While these feature are independent, you would expect a high correlation between mean and median.

Do not confuse this with the kind of PCA that is performed on databases of images. In that case, each image is an observable (what Image Analyst called a "feature" above). If you have 100 gray scale images, you can do PCA on the collection of those images as a whole. In this case, the dimensionality is equal to the number of pixels in the images, and the images must be normalized to the same number of pixels(by resizing usually).

[Walter Roberson]

I have to disagree on the "Answer: no" of #2. As you explored in #3, feature extraction can be done over the single image and the resulting features can be PCA'd: that is a "meaningful" way of doing PCA on a single grayscale image. The features can be extracted in multiple ways, including some ways that come down to geometric subsets. It would be uncommon that vertical quarters would have useful information but finer grained localities certainly might.

For example, if an image happens to be a multiple of 8 pixels horizontally and vertically, then break down the image into 8 x 8 sub-blocks and reshape() that into N x 64 and pca() that. The information so obtained about how the pixels in the block tend to relate to each other can be used for image compression.

More generally, breaking into P x Q blocks, permute and reshape to N x (P*Q), PCA, and the information you get out can be interpreted as texture information; you can probably vary P and Q over a range to get an idea of what the texture size is. (Though perhaps an FFT approach might be easier for that, I am not sure.)

pca() of an entire grayscale image without reshaping or extraction could potentially be meaningful for detecting texture.

[Image Analyst]

But Walter, that would be creating multiple features from an image, like measurements, or chunks of the image. And we agree with that - if you derive the right information from a single image, it can mean something. But if you don't, it's meaningless.

But I think Stefan was referring to doing PCA on just a single gray level image where the only feature is the gray levels - in that case you'd have a single PC and that is the gray level itself. If you divide the image up into meaningful regions like strips of 8 pixels wide, and that corresponds to some pattern in the image, then it might mean something. But if you're not smart about it and just arbitrarily divide, say, the cameraman demo image up into vertical strips 16 pixels wide, or the moon demo image up into quadrants, or the cell microscope image up into a checkerboard, then the PC of those things would probably not be meaningful -- you'd get something but it might not be interesting or useful to you. To be meaningful you have to create meaningful features from your input image in the first place. I know you'd agree with that.

[Walter Roberson]

If you take an arbitrary grayscale image and apply pca to the whole thing then you probably are not going to get much of interest out. But there are classes of images where the results might be interesting and useful. (Sometimes the useful bit of information would be that "this image is not a member of that class so move on to a different algorithm.")

The key is to have a reason to do the PCA you do, and to know what you are looking for in the results of the PCA.

The situation is not all that different from applying fft2 to arbitrary images: you are probably going to have a lot of difficulty understanding the results unless you know what you are looking for.

[Stefan Karlsson]

well.... I guess I will have to apologize for my inacuracies. You will get something useful from PCA over a single gray scale image:

you will get the global variance of the image, but that is the only thing. I do not disagree with anything that Walter says, it is technically correct.

Question: What is a Jumbo Jet made up of? I answer: mechanical parts, engines, fuel. While Walter answers by reference to the elements of the periodic table. What is the "right" answer depends only on the context the original question was posed...

(scale selection!) With that I leave this short but fun exchange. Maybe we should quickly note that PCA is actually not the best tool for what the question was about. I would guess Neo should use non-negative matrix factorization, or some other similar flavor. For photographic images, negative values are not allowed in neither basis nor coefficients....

sorry if i was harsh in my original reply.

[Image Analyst]

Nah, you were fine. But is a little frustrating waiting for Neo to verbalize what features he wants to characterize in his image. It's almost like he heard about PCA and thought that it sounded cool and wanted to apply it to his image without considering if it was appropriate or not, or whether there might be better methods. I still don't know what kind of result he would want.

Answer 4

[Stefan Karlsson]

... I guess one can also give another piece of advice to anyone who stumbles onto this looking for info on PCA. Read the posts by Image Analyst. They are as high in quality as they usually are.

[Neo]

I'd say Walter Roberson was also very high quality as well.

Answer 5

[Aya Ahmed]

Neo can you tell me please ..

i want to make feature extraction using PCA ,using matlab code on galaxy grayscale image ,

I was wondering if you could help me with a few steps or even code to make feature extraction from images .

I would like to extract the features of galaxy images and then classify them in the classification learner app.

The data I have is a set of galaxy imagse.

The aim is to extract the features and then compare them in the classification app with each other.

Any help is appreciated!

i want to know how pca work ? does it work in gray images only ??

Thanks ..