Fail to process multiselect files by using readtable function....
2 views (last 30 days)
Show older comments
Tyann Hardyn
on 15 Aug 2021
Edited: Tyann Hardyn
on 16 Aug 2021
Hi everyone. I just so confused about how to process and combine multifile datas by using readtable function. As you mention to my code below, iam using the readtable function because there is so many numheaderlines (consisted by random data type like numeric, string, etc) / heterogenous data which is should to be removed or skipped to get reach the header lines.
Format IAGA-2002 |%This headerlines should be removed or skipped
Source of Data Istituto Nazionale Geofisica e Vulcanologia |%This headerlines should be removed or skipped
Station Name Castello Tesino |%This headerlines should be removed or skipped
IAGA Code CTS |%This headerlines should be removed or skipped
Geodetic Latitude 46.047 |%This headerlines should be removed or skipped
Geodetic Longitude 11.649 |%This headerlines should be removed or skipped
Elevation 1175 |%This headerlines should be removed or skipped
Reported HDZF |%This headerlines should be removed or skipped
Sensor Orientation HDZ |%This headerlines should be removed or skipped
Digital Sampling 1 second |%This headerlines should be removed or skipped
Data Interval Type Filtered 1-minute (00:15-01:45) |%This headerlines should be removed or skipped
Data Type variation |%This headerlines should be removed or skipped
# MAGO 4.13.4 |%This headerlines should be removed or skipped
# SyS 1 |%This headerlines should be removed or skipped
# H, Z and F in nT, D in minutes of arc |%This headerlines should be removed or skipped
# F Digital Sampling 5 seconds |%This headerlines should be removed or skipped
# Approx H = 0 |%This headerlines should be removed or skipped
# Approx D = 0 |%This headerlines should be removed or skipped
# Approx Z = 0 |%This headerlines should be removed or skipped
# Vectorial Lemi 18 (text) - 115200 |%This headerlines should be removed or skipped
# Scalar GEM GSM 90 1 (048183.03 09) - 9600 |%This headerlines should be removed or skipped
# Scalar Tuning = 47 |%This headerlines should be removed or skipped
DATE TIME DOY CTSH CTSD CTSZ CTSF |%This is my header of data column i want to use
2021-07-05 00:00:00.000 186 22495.32 6.92 42278.17 47880.43
2021-07-05 00:01:00.000 186 22494.96 6.88 42278.31 47880.39
2021-07-05 00:02:00.000 186 22495.40 6.91 42278.35 47880.62
2021-07-05 00:03:00.000 186 22494.58 6.97 42278.27 47880.19
2021-07-05 00:04:00.000 186 22494.71 6.90 42278.35 47880.30
2021-07-05 00:05:00.000 186 22495.76 6.83 42278.47 47880.94
2021-07-05 00:06:00.000 186 22495.51 6.84 42278.35 47880.79
2021-07-05 00:07:00.000 186 22495.11 6.98 42278.14 47880.40
2021-07-05 00:08:00.000 186 22495.22 6.99 42278.19 47880.45
And by using a single file input (uigetfile) without multiselect function, iam trying to create a script to get data column of each of those header (DATE, TIME, DOY, CTSH, CTSD, CTSD, CTSZ, and CTSF) and it worked. However, i just wonder, how to manage a new script by using the readtable concept so that i can get the output file just like the single file script one. Iam trying to comprehend the document and write down the multiselect script, but it still doesnt success.... Here is my trial and error script :
[if file > 1
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
namas = namafile{:,1:k}
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
per = strcat(string(T2{1,1}));
tahun = str2double(extractBetween(per,1,4));
bulan = str2double(extractBetween(per,6,7));
hari = str2double(extractBetween(per,9,10));
D = datetime(tahun,bulan,tanggal,0,0,0,'TimeZone','Asia/Jakarta');
wak = DATE +" "+ TIME;
ntropy = string(wak);
DOH_combine(:,k) = DOH(:);
DOHn = reshape(DOH_combine,[],1);
DOFk = sqrt((DOH.^2)+(DOZ.^2));
DOFk_combine(:,k) = DOFk;
DOFk_combine(DOFk_combine == 0)=[];
DOFn = reshape(DOFk_combine,[],1);
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+namas;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
end
if judul2 == 15 || varm == 1
for n = 1: nfiles
[savefile, direct, ~] = uiputfile('*.txt', 'Save As', sti);
eval(['cd ''' direct ''';']);
fout=fopen(savefile,'w');
for i=1:length(TIME)
DOFS = sqrt((DOH.^2)+(DOZ.^2));
fprintf(fout,'%s %s %.2f %.2f\n', char(DATE(i)), char(TIME(i)), DOH(i), DOF(i));
end
fclose(fout);
end
figure('Name','Plot Of Horizontal Magnetic Field IAGA Format Vs Time Series')
plot(wak, DOHn, 'Color', 'r', 'LineWidth',1.1);
title(th, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen H Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
figure('Name','Plot Of Total Magnetic Field IAGA Format Vs Time Series ')
plot(wak, DOFn, 'Color', 'b', 'LineWidth',1.1);
title(tf, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen F Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
I just confused how to manage data table which re acquired from multiselect readtable onto another file format that can be processed. The error is almost always like this :
Error using teslemiiaga (line 232)
Right hand side of an assignment into a table must be another table or a cell array.
So... Would anyone lend me a hand to fix the error of the above script i made, or maybe just fix the entire concept of my script? and yes, i just wonder how it could be fixed by another function besides of readtable, maybe like readmatrix or readcell? Thank you everyone..... (i already attached my data).
Iam sorry for interrupting your days /.\ /.\ /.\
0 Comments
Accepted Answer
Walter Roberson
on 15 Aug 2021
Edited: Walter Roberson
on 16 Aug 2021
When you have code that processe only one file from a uigetfile(), then it is of the form
[filename, directory] = uigetfile(appropriate parameters);
if isempty(filename)
return; %user cancel
end
fullname = fullfile(directory, filename);
Now do code processing the one file recorded in fullname
When you have multiselect turned on, then instead it is of the form
[filenames, directory] = uigetfile(appropriate parameters);
if isempty(filenames)
return; %user cancel
end
filenames = cellstr(filenames); %takes care of case only one file was selected
fullnames = fullfile(directory, filenames);
all_tables = {};
for K = 1 : length(fullnames)
fullname = fullnames{K};
Now do code processing the one file recorded in fullname
all_tables{K} = output table;
end
combined_table = vertcat(all_tables{:});
save combined_table in appropriate place
6 Comments
More Answers (1)
Simon Chan
on 15 Aug 2021
Not quite understand your issue, but I guess you are not able to get the figure when the code is reading multiple files.
So I put the code which generate figures inside the loop. If you would like to save different files in similar way, you may also put the code inside the loop.
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
namas = namafile{:,1:k};
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
per = strcat(string(T2{1,1}));
tahun = str2double(extractBetween(per,1,4));
bulan = str2double(extractBetween(per,6,7));
hari = str2double(extractBetween(per,9,10));
D = datetime(tahun,bulan,hari,0,0,0,'TimeZone','Asia/Jakarta'); % Modify the avriable to 'hari'
wak = DATE +" "+ TIME;
ntropy = string(wak);
DOH_combine(:,k) = DOH(:);
DOHn = reshape(DOH_combine,[],1);
DOFk = sqrt((DOH.^2)+(DOZ.^2));
DOFk_combine(:,k) = DOFk;
DOFk_combine(DOFk_combine == 0)=[];
DOFn = reshape(DOFk_combine,[],1);
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+namas;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
%
% Put the figure generation iside the loop
%
figure(k)
subplot(2,1,1)
%figure('Name','Plot Of Horizontal Magnetic Field IAGA Format Vs Time Series')
plot(datetime(wak), DOH(:), 'Color', 'r', 'LineWidth',1.1); % Convert to datetime and select one column of data
title(th, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen H Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
subplot(2,1,2)
%figure('Name','Plot Of Total Magnetic Field IAGA Format Vs Time Series ')
plot(datetime(wak), DOFk_combine(:,k), 'Color', 'b', 'LineWidth',1.1); % Convert to datetime and select one column of data
title(tf, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen F Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
end
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/712472/image.jpeg)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/712477/image.jpeg)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/712482/image.jpeg)
3 Comments
Simon Chan
on 16 Aug 2021
As i mentioned earlier, you can put the save file section inside the loop as well.
Please refer to the following as a reference. You can actually rearrange your code to achieve your target.
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
%namas = namafile{:,1:k}; % No need this line
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
% Rearrange the following lines before you write the txt file
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+nama;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
if judul2 == 15 || varm == 1
[savefile, direct, ~] = uiputfile('*.txt', 'Save As', sti);
eval(['cd ''' direct ''';']);
fout=fopen(savefile,'w');
for i=1:length(TIME)
fprintf(fout,'%s %s %.2f %.2f %.2f %.2f\n', DATE(i), TIME(i), DOH(i), DOD(i), DOZ(i), DOF(i));
end
fclose(fout);
end
end
See Also
Categories
Find more on Tables in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!