I keep getting NaN values when computing the mean and standard deviation of an array

Hi, I am attempting to calculate the mean of all the rows in a cell array called 'XTrain2_all' comprised of 4x30 doubles, however I keep getting NaN values instead of 4 mean values. The same happens when computing the standard deviation. How do I stop this from happening? Thanks in advance.

XV = [XTrain2_all{:}];
mu = mean(XV,2);
sg = std(XV,[],2);

Answer 1

[Walter Roberson]

Each of your rows has 30 NaN values.

In rows 1 and 2, they are in columns 95221 to 95250.

In rows 3 and 4, they are in columns 94951 to 94980 .

mean() of a value that includes nan is going to be nan, because anything added to nan gives nan as a result.

You can use

XV = [XTrain2_all{:}];
mu = mean(XV, 2, 'omitnan');
sg = std(XV, [], 2, 'omitnan');

[Cai Chin]

Hi Walter, thanks very much. Would it instead be possible to interpolate the NaN values using the framing values? Thanks

[Steven Lord]

See the fillmissing function.

[Cai Chin]

Hi, when I use this function with the 'linear' method, it gives the following error:

First argument must be numeric, logical, datetime, duration, calendarDuration, string, categorical, char, cellstr, table, or timetable.

But I need this variable to be in a cell array format. Moreover, could you please advise me as to how to instead replace the 30 NaN values with a replicate of the previous 30 missing values instead of doing a linear interpolation since the data does not follow a linear trajectory. Thanks

[Walter Roberson]

Do the fillmissing on XV .

[Cai Chin]

Hi, it's just that I need 'XTrain2_all' without missing values since XV is only used to calculate the mean and standard deviation, after which these are applied to 'XTrain2_all' and another variable as shown below. Also, would it be instead possible to instead replace the 30 NaN values with a replicate of the previous 30 missing values instead of doing a linear interpolation since the data does not follow a linear trajectory?

XV = [XTrain2_all{:}]; % Need to remove NaN values and replace by inetrpolation
mu = mean(XV, 2);
sg = std(XV, [], 2);

XTrainSD = XTrain2_all;
XTrainSD = cellfun(@(x)(x-mu)./sg,XTrainSD,'UniformOutput',false);

XTestSD = XTest2_all;
XTestSD = cellfun(@(x)(x-mu)./sg,XTestSD,'UniformOutput',false);

[Steven Lord]

x = [1:5 NaN 7 NaN(1, 4)]

x = 1×11

1 2 3 4 5 NaN 7 NaN NaN NaN NaN

y = fillmissing(x, 'previous')

y = 1×11

1 2 3 4 5 5 7 7 7 7 7

●

[Cai Chin]

Hi Steven, in my situation, the NaN values all follow eachother across 30 values. Therefore, instead of replacing by the previous number, is it possible to replace these 30 values with the 30 values prior to these missing ones rather than just replacing with the last non-missing value?

[Walter Roberson]

Just loop. It isn't worth overthinking the situation.