How to open files in different folders for data processing?

Hello! I'm currently tasked with plotting data from several different files that are in different folders due to the way the data was processed. How can I script matlab to look through different folders and get a file (the file in each folder has the same name but the folders have different names) and grab the data that I am interested in?

I am very new to matlab so this is hard for me to try to do.

Thank you,

-Levi

Answer 1

[Bob Thompson]

Take a look at this.

https://www.mathworks.com/matlabcentral/fileexchange/172-findfiles

It is possible to look through directories recursively with newer versions of matlab using the dir command, but the above should work for all relatively common versions in use.

[Levi Wierman]

So would this work for the folders under the same directory if each folder is something of the form "filename_A, filename_B, filename_C... filename_M?" And how would I do that? I never really learned to code in C++ to search for things like that so I am totally unfamiliar with how to do that.

[Bob Thompson]

The link presented is for a matlab code. Basically, you feed it a parent directory (the highest level that contains subdirectories with the files you want) and a string check (something that relates to your files). It will then find all files with that string in the parent directory, and any subdirectories within the parent directory (no matter how many levels deep). The results will be a list of file names, similar to what you get from 'dir'.

files = findfiles('myfile.csv','C:/Users/Me/Work');

From there you can just run a loop through the list of files and conduct whatever processing you need on each one.

[Levi Wierman]

I keep getting this error:

Invalid expression. When calling a function or indexing a variable, use parentheses.

Otherwise, check for mismatched delimiters.

I am not sure how to fix it. I've tried several things. As I said, I'm very new. This is my line of code to call the function:

datamatrix=findfiles('discharge_data.dat','C:\Users\LeviW\Documents\PlasmaData',);

I left the recurse part empty but I have also tried typing 1 and (1). I'm not getting anything.

And when it does not give me that error, it gives me this:

Cannot find an exact (case-sensitive) match for 'findfiles'

The closest match is: FindFiles in C:\Users\LeviW\Documents\MATLAB\FindFiles.m

discharge_data.dat is the file name that I am trying to assemble into a single matrix but the file is throughout a bunch of different folders under the "PlasmaData" directory. I need to assemble all the data into one matrix in a specific order, as I'm sure I have specified multiple times now, I am just getting frustrated. I really appreciate your help

[Jan]

You cannot leave an input argument empty in Matlab:

datamatrix=findfiles('discharge_data.dat', ...
    'C:\Users\LeviW\Documents\PlasmaData',);
%                                        ^^

This is not allowed. You can omit a trailing input argument, but then the comma is left out also. See the examples included in the function.

Matlab is case-sensitive and as the error message tells clearly, the function is called "FindFiles" with uppercase Fs.

Why don't you use the dir function, which works recursively since R2016b?

[Levi Wierman]

Because I don't understand how to make it work. The syntax looks like it needs a direct file path to the file I need to load. But LIke I said, I have 10 files in 10 different folders I need to access and I'm not sure how to make that work.

[Jan]

I've posted a solution 2 days ago already. So I ask again:

Why don't you use the dir function, which works recursively since R2016b?

[Levi Wierman]

I will say it again, I do not know how to make it work. You gave me a solution that looks like it requires me to give matlab the path directly to the file but that is not what I need to do... because it's in a bunch of folders. I am brand new to matlab and I have a hard time understanding minimalist syntax exmples in the operators. Is filename an array that has each file location or whatever in it? What is 1:nume1?

[Jan]

@Levi: If you ask for details about my answer, please post them as comments to my answer.

The \*'*\ is the search string of the dir command means a recursive search in all subfolders.

Please take the time to look into th documentation:

help numel

The last character is an lowercase L, not a one.

You cannot learn the basics in the forum. The Getting Started chapters of the documentation and the "Onramp" (ask an internet search engine) are more efficient.

You can simply run the code and check, what the contents of the variable FileName is.

Answer 2

[Jan]

Since Matlab >= R2016b:

List = dir('C:\Your\Data\Folder\**\FileName.txt');
FileName = fullfile({List.folder}, {List.name});
for k = 1:numel(FileName)
    disp(FileName{k})
    ...
end

[Levi Wierman]

I wasn't aware of the ** being the command to search recursively, I thought it was a shortcut to basically say "however long the path is." I'll try to make the dir command work now that I do not have a deadline looming!

Thank you.

[Stephen23]

"I wasn't aware of the ** being the command to search recursively, I thought it was a shortcut to basically say ..."

tip for the future: guessing how MATLAB works is very inefficient. Reading the documentation is much better (and if something is not clear, as Jan already wrote, you can ask us to clarify).