Replace NaN by using for loop

I have a set of data (call it 'dataset') which is a 1000*1000 table with NaN, NaN appear start from row 381 to the end. I design to use for loop to find the mean number from row 1 to row 380 in each column and use it to replace the NaN in that column. I have write the condition but get stuck for the loop because I have no idea code the loop.(I'm a matlab beginner)

dataset = table2array(dataset);     %convert table to matrix for modify data
columns = size(dataset,2);
for i = 2:1:columns - 1        %start from second column, end at second last column
  MeanNo = mean(dataset(1:380,i));      %get the mean number from row 1 to row 380 for one column
  dataset(isnan(1:end,i)) = MeanNo;     %replace NaN with the mean number just got
end
dataset = array2table(dataset);           %convert back to table

That is my code and it cannot be run, I have no idea how can I solve it

Answer 1

[Star Strider]

One approach:

dataset = [randi(9, 10, 5); NaN(10, 5)];                            % Create Data
dataset_mean = mean(dataset, 'omitnan');                            % Column Mean Omitting ‘NaN’ Values
dataset_new = dataset;                                              % Create Duplicate
idx = isnan(dataset_new);                                           % NaN == 1, Others == 0
dataset_new(idx) = 0;                                               % Set ‘NaN’ Values = 0
dataset_new = dataset_new + bsxfun(@times, idx, dataset_mean);      % Add ‘Replacement’ Values To ‘dataset_new’

[Joe Sun]

My dataset have a lot of 0 and NaN. Will those value 0 be replaced for your approach?

[Star Strider]

The 0 values will not be replaced. The NaN values will. I went back and tested my code with this matrix to be sure:

dataset = [randi([0 9], 10, 5); NaN(10, 5)];                        % Create Data

[Joe Sun]

thank you very much for the helping!

[Star Strider]

As always, my pleasure!

Answer 2

[Matthew Eicholtz]

When you say "it cannot be run", do you mean that you receive an error? If so, what line does the error point to and what is the error description? I'm guessing you may get an error for the following line of code:

dataset(isnan(1:end,i)) = MeanNo;

I think it should be:

dataset(isnan(dataset(1:end,i))) = MeanNo;

A couple other tips that may be helpful for the future:

1. You can replace

2:1:columns - 1

with

2:columns-1

because the default incremental value for the colon operator is 1.

2. You can compute the mean of each column in a matrix in one operation. So, if you know you will want the mean of rows 1:380 for every column, it can be computed by:

m = mean(dataset(1:380,:));

*If you wanted the mean of every row, you could use:

m = mean(dataset(1:380,:),2);

Answer 3

[Guillaume]

The simplest way to get rid of NaNs in a table is to use the fillmissing function. In your case,

fillmissing(yourtable, 'movmean', 1000)

would probably give you what you want.