what does eigenvalues expres in the covariance matrix?

is there a relationship between a covariance matrix and eigenvalues? like an example

Let us consider a 321 × 261 image dimention 321 × 261 = 83781. We have only 32 observations and 83781 unknowns then we have a matrix of (32 row X 83781 column)

then we will calculate the covariance matrix (32 X 32) so we get 32 eigenvalues the question is: does these eigenvalues express the 32 images? or there is no any relationship between eigenvalues and images

thanks for you,

[Matt J]

No, the covariance matrix should be 83781x83781, not 32x32.

[Cedric]

http://www.mathworks.com/matlabcentral/answers/73162-is-there-a-relationship-between-eigen-values-and-covariance-matrix

[Mohamed Moawed]

how??? could you please give me some details

[Bikramjit Singh]

32 are observation (samples), so covar should be 83781x83781.

You can assume 8371 are components; eigenvalues (so you get 8371 eigenvalues), which can be interpreted as energies along those components, so sum of eigen values = total energy (this what i understood from signals point of view)

[Mohamed Moawed]

ok, now the covariance matrix is 83781 X 83781 then number of eigenvalues equal 83781

my question, does each eigenvalue expresses each row in the covariance matrix or expresses each row . or there is no relationship between each others.

[Bikramjit Singh]

there is no such row or column thing, i mean you can't interpret eigen values directly.

But, as I said total energy = sum of eigen values = trace i.e sum of diagonal values of covar matrix

let H=83781x32 sized matrix

Therefore, Covar matrix, C=H*H' = of size of 83781 X 83781

sum(eig(C)) = trace(C)

[Mohamed Moawed]

what about image point of view??? when i want to select some eigenvectors.how can i say this eigenvector for this observation or for this image

[Vincent Spruyt]

The eigenvalues in this case represent the magnitude of the spread in the direction of the principal components. If you data has a diagonal covariance matrix (covariances are zero), then the eigenvalues are equal to the variances:

If the covariance matrix is not diagonal, then the eigenvalues still define the variance of the data along the the principal components, whereas the covariance matrix operates along the axes:

Here is an article (and the source of the above images) that discusses this in more detail: http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/

Answer 1

[Kye Taylor]

Long story short: The eigenvalues of the covariance matrix encode the variability of the data in an orthogonal basis that captures as much of the data's variability as possible in the first few basis functions (aka the principle component basis).

For example, this code creates an ellipse, whos major axis is the x-axis, and whos minor axis is the y-axis.

t = linspace(0,2*pi,256);
data = [cos(t);0.2*sin(t)];
plot(data(1,:),data(2,:),'.')
axis([-1,1,-1,1])

Now, compute the variance of the data's coordinates

% tranpose to get variance down each column
% computes variance of each coordinate of the data
v = var(data')

Finally, observe the eigenvalues of the covariance matrix are equal to the variance of the data's coordinates

% need to transpose since input to cov must have
% rows = observations
% columns = variables
l = eig(cov(data'))

In other words, v and l contain the same values. The order may be different because the eig function returns eigenvalues in no particular order.

Note that if the data is rotated so that the major and minor axes are no longer the x and y axes, then var(data') no longer computes the variance about the coordinates from their principle axes, but eig(cov(data')) does automatically since the eigenvectors are principle components.

For example

% rotate everything
r = [cos(pi/4),-sin(pi/4);sin(pi/4),cos(pi/4)]; % rotation matrix
rData = r*data;

hold on
plot(rData(1,:),rData(2,:),'r.') % plot rotated data

vr = var(rData') % different
lr = eig(cov(rData')) % should be same as l

[Mohamed Moawed]

what about images???

[Kye Taylor]

It sounds like you're asking a different question that what the main thread is, and what I answered above but here's a nice little example getting at what you might be interested in .. If you can, give me more insight into what you're trying to do?

load mri.mat % included w MATLAB
montage(D,map) % 27 images in mri.mat

% organize the data into a nImages-by-nTotalPixelsPerImage array
D = permute(D,[4,1,2,3]);
data = reshape(D,[size(D,1),size(D,2)*size(D,3)]);

% do something crazy, and compute the first 12 
% eigenvectors of this 2^14-dimensional matrix
[v,d] = eigs( cov(double(data)), 12,'LA');
d = diag(d);  % only the eigenvalues
maxD = max(d);

This figure contains the principle components of your data, scaled according to the variance along that component. I know, it's cool.

figure
for j = 1:12
    subplot(4,3,j)
    imagesc(reshape(v(:,j)*d(j)/maxD,siz(1),siz(2)))
    colorbar
    colormap gray
end

If you look at a plot of the eigenvalues, notice that more than half of the variance in the data is captured in the first three principle components

figure,plot( d/maxD, '.-')

[Mohamed Moawed]

wow, that is nice, thanks for your kindly reply. i need more thing, in my work i have several fMRI brain images for Alzheimer Disease. and i get these images and compare between each part in these images to get the parts that are different between positive and negative images. how can i get these parts? i want to use PCA to get eigenvalues that they are the principle component of these images but how can i say this eigenvalue for this part in this image?

[Kye Taylor]

Open up a new thread... and kindly accept my answer.

Answer 2

[Shashank Prasanna]

Essentially what you are describing are the principal components of your data.

https://en.wikipedia.org/wiki/Principal_component_analysis#Computing_PCA_using_the_covariance_method

Its a popularly used dimensionality reduction technique, for example to make your image smaller such that it still retains most of its variance.

The PCA command in MATLAB does all this for you directly.

http://www.mathworks.com/help/stats/pca.html

[Mohamed Moawed]

yes i know i want to use PCA . but i dont understand how to use it with more than one image to get the principle component of these images

[TUSHAR MURATKAR]

In my reference paper related to wireless communication the covariance matrix is made from vector comprising of channel coefficients. And non zero eigen values of the covariance matrix are calculated. What does this signify with reference to wireless communication.