Opening multiple .csv files using readtable

Hi there

I am having trouble trying to create a script that will open up all .csv files in a subfolder and then save each of those as a .mat file. I have a few hundred of these individual .csv files and find it much easier to analyse information in matlab when the data is a .mat file so am trying to find a way to speed up the process rather than using the small code I have and having to select the individual file each time. If I want to change just one file I use the following that I generated from the import tool in Matlab (which works). Each of the files have the same number of variables and the same variable names. I'm hoping someone is able to help me figure out how to loop this so that it will run through all the files. I have read the help information in Matlab on for loops and have searched in Matlab Answers for help, but I have not been able to figure this out myself and would really appreciate any assistance.

%Import options

opts = delimitedTextImportOptions("NumVariables", 4);

% Specify range and delimiter

opts.DataLines = [3, Inf];

opts.Delimiter = ",";

% Specify what the column names are and what the variable types are.

opts.VariableNames = ["Time", "Displacement", "Force", "TensileStress"];

opts.VariableTypes = ["double", "double", "double", "double"];

% Specify file level properties

opts.ExtraColumnsRule = 'ignore';

opts.EmptyLineRule = 'read';

% Import the data from the file named

SLC11_1 = readtable("C:\Users\krist\OneDrive\Documents\MATLAB\SLC11_1.csv", opts);

% This will show the read .csv file as a table and then save it in the .mat format

SLC11_1 = table2array(SLC11_1);

save('SLC11_1.mat')

Answer 1

[Stephen Cobeldick]

% opts is unchanged
D = 'C:\Users\krist\OneDrive\Documents\MATLAB';
S = dir(fullfile(D,'*.csv'));
for k = 1:numel(S)
    F = fullfile(D,S(k).name);
    T = readtable(F, opts);
    A = table2array(T);
    [~,G] = fileparts(F);
    save(sprintf('%s.mat',G),'A')
end

[Kristy Garrick]

Thank you so much for your speedy help with this! It works perfectly :) While I may not know what all of it means at least now I can work backwards from your script and learn what everything is. Thank you again!

[Kristy Garrick]

Hi again, just a quick follow on question - is there anyway to make this function save the Varialble the same as the file name? I'll be doing a lot of plotting with these and at the moment with each file I load it the variable is always 'A' in the workspace and will overwrite this.

[Stephen Cobeldick]

" is there anyway to make this function save the Varialble the same as the file name?"

Yes, but I do not recommend that approach: forcing meta-data (such as a filename or index number) into variable names makes processing data slow, complex, inefficient, obfuscated, liable to bugs, and difficult to debug.

"I'll be doing a lot of plotting with these and at the moment with each file I load it the variable is always 'A' in the workspace and will overwrite this."

You can avoid this by loading into an output variable. The recommended approach is to load into one variable using indexing, exactly as the MATLAB documentation shows:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

For example, loading one variable (which always has exactly the same name) from multiple .mat files:

D = 'path to where the files are saved';
S = dir(fullfile(D,'*.mat'));
for k = 1:numel(S)
    F = fullfile(D,S(k).name);
    T = load(F); % LOAD file data into an output variable!
    S(k).data = T.A; % store/do whatever with the imported data
end

The imported variable data is stored in the structure S. You can trivially index into S to access the data, e.g. the 2nd file's imported data (obviously you can change the fieldname/s to suit your needs):

S(2).data

or use a comma-separated list to operate on subset/all of the data, e.g. if all of the imported data arrays have compatible sizes:

alldata = horzcat(S.data)

https://www.mathworks.com/help/matlab/matlab_prog/comma-separated-lists.html

https://www.mathworks.com/matlabcentral/answers/320713-how-to-operate-on-comma-separated-lists