Remove rows with NaNs from 2 variables to do a corrcoef and a polyfit
Show older comments
Hi all! Since polyfit and corrcoef do not remove NaNs I am trying to remove NaNs first and then do the correlation.
The situation is the following: I have two variables, one is the observed and the other is the forecasted one. Both of them have 17 columns (each forecast time) and 9500 lines aprox.
What I know is that I need to ask for NaNs in at least one of them and then remove that row.
I tried this (my variables are rh_media_3 and hr_ref2):
for i=1:size(rh_media_3,2)
tmp= [ rh_media_3(:,i) hr_ref2(:,i) ];
%rh_valid=rh_media_3(~any(isnan(tmp),2),:);
%hr_ref2_valid=hr_ref2(~any(isnan(tmp),2),:);
rowsWithNaN = any(isnan(tmp),2);
rh_media_3=rh_media_3(~rowsWithNaN,:);
hr_ref2=hr_ref2(~rowsWithNaN,:);
end
for i=1:17
tmp=polyfit(rh_media_3(:,i),hr_ref2(:,i),1);
pendiente_hr(i)=tmp(1);
temp=corrcoef(rh_media_3(:,i),hr_ref2(:,i));
coef_lineal_hr(i)=temp(2,1);
end
No matter what I do I get coefficients greater than 1, which is not real. Can anyone help please?
Accepted Answer
More Answers (1)
dpb
on 19 Jul 2013
0 votes
If the concern is over the slopes of the linear fit, there's nothing that says limits its magnitude; that'll be wholly dependent upon the way the response variable (hr_ref2) varies w/ the independent (rh_media_3).
If the forecast response is preferentially biased high against the observed value, that's exactly what one would expect.
Plotting the data and the fitted line would probably reveal a lot.
2 Comments
Tamara Schonholz
on 19 Jul 2013
Say what? The slope of what says anything at all about the variance contributions? The R-square value does (sorta') say that, but certainly not the slope--all it does is reflect the slope (duh) of the least-squares fit line of the two vectors.
Plot it and see; I'm sure it'll be obvious for your data that the slope is >1 owing to the way the data fall out...now why that is so if it isn't what you think should be is another question entirely.
See Answer section for how to compute R-sq from coefficients and data.
Categories
Find more on Correlation and Convolution in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!