How to interpret the covariance matrix elements: diagonal versus off-diagonal terms

Dear all, do you know how to read a normalized covariance matrix? Because it is normalized all diagonal elements are "1". I do have off-diagonal elements greater than "1". How is it possible? I thought that cov(i,j) was the level of similarity of vector_i with vector_j. If this is the case, then off-diagonal elements must be less that "1". (???)

I do not think that it is relevant, but (just in case) I am working on face recognition and the covariance matrix is the one of some ordered coefficients used to classify the images, every class has 30 samples. I do calculate the 30*30 covariane matrix of 1102 samples of dimension 30. All vectors are normalized (I imposed unit norm by column), and also the covariance matrix is normalized (I impose "1" on the diagonal).

Thank you, Idil

[Wayne King]

How are you "normalizing" your covariance matrix, please show the relevant MATLAB code, because cov() doesn't do that.

Answer 1

[Greg Heath]

I need to factor the covariance matrix to get a feeling for the properties and associations of the individual variables.

Consider the variables x and y with standard deviations 2 and 3 and correlation coefficient 0.6. The corresponding matrices are (cut and paste into the command line)

STDxy = [ 2 0 ; 0 3 ] CORxy = [ 1 0.6 ; 0.6 1 ]

% the corresponding covariance matrix is

COVxy = STDxy*CORxy*STDxy

COVxy = [ 2^2 2*3*0.6 ; 2*3*0.6 3^2 ]

% Conversely,

STDxy = sqrt(diag(diag(COVxy)))

CORxy = inv(STDxy)*COVxy*inv(STDxy)

Hope this helps.

Greg P.S. If one of the standard deviations is zero, replace inv with pinv.