MATLAB Answers

How to convert time into nearest hour and to do interpolation of NaN values extracted from .csv file?

4 views (last 30 days)
Preyanka Dey
Preyanka Dey on 13 Mar 2020
Commented: Guillaume on 17 Mar 2020
Hi,
I have a .csv file from which I extracted values of three columns. Now I want to convert time into nearest hour and interpolate NaN values of last column named 'prec'. I tried to round time using 'dateshift'. However, it is not working. I also tried to fill the missing values. But, it is returning negative values which are supposed to be something else. Can anyone help me please? Thanks
function main
fid = fopen('see.csv','r')
c = textscan(fid,'%s %s %s','delimiter',',');
fclose(fid);
date = c{:,1};
time = c{:,2};
prec = str2double(c{:,3});
Rtime = dateshift(time,'start','hour') + minutes(00); % not working
Fprec = fillmissing(prec,'spline',10,'endvalues','nearest'); % returnig negative values which are supposed be something else
end

  0 Comments

Sign in to comment.

Accepted Answer

Guillaume
Guillaume on 13 Mar 2020
Here's how I'd import your data.
opts = detectImportOptions('see.csv'); %let matlab figure out the format of the file.
opts = opts.setvaropts(1, 'Type', 'datetime', 'InputFormat', 'uuuu/MM/dd', 'DatetimeFormat', 'dd/MM/uuuu HH:mm:ss'); %however matlab doesn't detect the first input as datetime
opts.VariableNames = {'DateTime', 'Time', 'Something'}; %give the variable names some more useful names. If the csv file had a header matlab would automatically use it to name the variables
something = readtable('see.csv', opts); %import as table. Obviously use a different name for the output.
something.DateTime = something.DateTime + something.Time; %merge date and time into one datetime
something.Time = []; %and get rid of the time column now that it is part of the date
something.DateTime = dateshift(something.DateTime, 'start', 'hour', 'nearest'); %shift to the nearest hour
something.Something = fillmissing(something.Something, 'linear');
With regards to the fillmissing, considering your numbers are all near 0, it's no wonder you get negative numbers when using cubic interpolation. If you don't want that, use a different fill method.
However, are you sure that you really want to round the time to the nearest hour without affecting the values in the 3rd column? Instead you may want to convert the table to a timetable (at the step before the dateshift, then retime the timetable to hourly:
%.. same code as before up to:
something.Time = [];
something = table2timetable(something);
something = retime(something, 'hourly', 'spline'); %or some other interpolation method

  4 Comments

Show 1 older comment
Guillaume
Guillaume on 17 Mar 2020
Again, you're not using the tools that matlab give you to make these tasks much easier. You can use readtable as I've shown to read the file directly as required. Even better in this case, use readtimetable:
opts = detectImportOptions('2015_NOAA.csv', 'Delimiter', ','); %Matlab autodectect as much of the format as possible. It still doesn't detect the time right
opts = opts.setvaropts('DATE', 'Type', 'datetime', 'InputFormat', 'uuuu-MM-dd''T''HH:mm:ss'); %so override the decoding of that
precipitation = readtimetable('2015_NOAA.csv', opts); %read the file directly as timetable
This gives you an easy to use timetable with your file header used to name the variables of the time table. You then have many lines of code to basically chose the largest precipitation value of each hour (this is what your sortrow and unique(.., 'last') end up doing). Again, there's a much simpler way to do that with retime:
hourlyprecipitation = retime(precipitation, 'hourly', 'max'); %take the maximum precipitation of each hour
And writing to another text file with a space delimited is just as easy with writetimetable:
writetimetable(hourlyprecipitation, 'my_output2.txt', 'Delimiter', ' ', 'WriteVariableNames', false);
Preyanka Dey
Preyanka Dey on 17 Mar 2020
Thanks Guillaume. The second line is not working. Howerver, I created a timetable and your mentioned 'writetimetable' (last line) also worked. Actually, it has been only two months that I am using MATLAB. That's why I am not familiar with most of the tools. Anyway, again thank you very much for your help and patience.
Guillaume
Guillaume on 17 Mar 2020
What error message do you get (please give the full message).
Also which version of matlab?
I tested the code with your attached file and get no error.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!