How to set a condition for sequential number of non NaN values in a vector?

Hello, I have a matrix like that:

NaN  -0.00042   0.00073   -0.00298  NaN
NaN  -0.00114   0.00029   0.00265  NaN
NaN  -0.00028   NaN       -0.00066  NaN
NaN  0.00197     0.00219   0.00099  NaN

I would like to write a condition for each column: if the number of SEQUENTIAL non-NaN values >=300 (no matter where exactly in a column and non NaNs have not be the same, just any non NaNs), then leave the column, otherwise - delete the whole column. I'm stuck with that "sequential" - how can I say to matlab look for 300 consecutive non-NaNs in a row??

I would appreciate any help.

[dpb]

Gotta' run; realized I'm late for a meeting...but look for a "runs" submission on FileExchange (maybe Bruno, I forget???).

Answer 1

[Roger Stafford]

Let M be your matrix.

   [m,n] = size(M);
   T = diff([true(1,n);isnan(M);true(1,n)],1,1);
   T2 = false(1,n);
   for k = 1:n
     f = find(T(:,k));
     T2(k) = any((f(2:2:end)-f(1:2:end-1))>=300); % <-- Corrected
   end
   M2 = M(:,T2);

M2 will be your result. I assume that where you said "delete" you meant that literally - such columns should be removed from the matrix.

[Ekaterina Serikova]

Roger, thanks a lot for your reply! Could you please explain what is the second "true(1,n)" is exactly doing in the second line

T = diff([true(1,n);isnan(M);true(1,n)],1,1)?

[Roger Stafford]

Suppose you have a column with values c = [1;2;3;Nan;4;5]. If I do

   diff(true;isnan(c);true)

I get [-1;0;0;1,-1;0;1] and when I do find, the result is

   f = [1;4;5;7];

so that f(2:2:end)-f(1:2:end-1) = [3;2] which are the correct numbers of consecutive non-nans. If I hadn’t appended that final true(1,n), this computation would have failed.

[Ekaterina Serikova]

Thank you very much!!

[Ekaterina Serikova]

Dear Roger, could you please advise how can I adjust the code for zeroes instead of NaNs? How can I Change isnan condition to zeroes? Thanks a lot.

[dpb]

Should be to just replace isnan(M) with M==0 in the first expression (although you may need to check on the initial condition as as long as there are no NaN in M the first value can't be but might it be 0?

Answer 2

[the cyclist]

Download RunLengths from the FEX.

Then run this code. (I used a length threshold of 3 for illustration. You'll want to use 300.)

LENGTH_THRESHOLD = 3;

M = [NaN  -0.00042   0.00073   -0.00298  NaN
NaN  -0.00114   0.00029   0.00265  NaN
NaN  -0.00028   NaN       -0.00066  NaN
NaN  0.00197     0.00219   0.00099  NaN]

numberColumns = size(M,2);

maxNonNanLength = zeros(1,numberColumns);
for nc = 1:numberColumns
    [b,L] = RunLength(M(:,nc));
    currentNonNanLength = 0;
    for nb = 1:numel(b)
        if isnan(b(nb))
            currentNonNanLength = 0;
        else
            currentNonNanLength = currentNonNanLength + L(nb);
            maxNonNanLength(nc) = max(maxNonNanLength(nc),currentNonNanLength);
        end
    end
end

hasSufficientNonNanLength = maxNonNanLength >= LENGTH_THRESHOLD;

M = M(:,hasSufficientNonNanLength);