Extract *.mat-Variables with same names from different directories

Hello,

before I even start an attempt to put my problems into words, take a loot at the attached screencap. It does a fine job of explaining the entire structure.

There are 96 of these directories in their parent folder 'J', all 96 of which are structurally named. However, I need to access the variable 'J' that lies within 'J.mat' in each of them.

I have an idea of iterating recursively through the folders (that's how I got them in the first place), but I am lost at the MATLAB code of loading them into the workspace. A simple 'load' command will give me a strut, which I can't do much with.

Ideally, I'd like a workspace containing all 96 variables structurally named, like 'J_333_01_a'. From there, I should be able to process them further (plotting etc). Thus far, I created arrays to function as a source for a potential for-loop:

 arrX=[333,343,353,363]; arrY=[0.025,0.05,0.1,0.2]; arrZ=['a','b','c','d','e','f']

I am stuck at actually translating my pseudo kind-of-bash-code into compatible MATLAB code:

x=[333,343,353,363];
y=[0.025,0.05,0.1,0.2];
z=['a','b','c','d','e','f'];
for xelement in x; do
  for yelement in y; do
      for zelement in z; do
          A=load('./xelement_yelement_zelement/J.mat');
          B=struct2cell(A);
          J_xelement_yelement_zelement=cell2mat(B)

The end goal is to compare the J values and draw conclusions about the influence of the parameters x, y and z. So converting from struts may not be ideal to reach that goal, but I am a complete noob so feel free to rectify.

Thank you!

Answer 1

[Stephen23]

"A simple 'load' command will give me a strut, which I can't do much with."

"Ideally, I'd like a workspace containing all 96 variables structurally named, like 'J_333_01_a'. From there, I should be able to process them further (plotting etc)"

Nope, you have that exactly the wrong way around. Loading data into lots of numbered (or otherwise uniquely named) variables would be the worst way of doing this. Loading into a structure and then either

1. storing the structure (could be non-scalar, if the fields are the same), or
2. extracting and storing the data within the structure

would be the fastest and most robust ways to do this. Read this to know why:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

http://www.mathworks.com/matlabcentral/answers/196952-how-to-create-new-variables-in-batches-with-strcat

The best solution is to take a deep breath, and learn to love structures. Really, you can do more with structures than you can by loading that data into independent variables. All of those things that you imagine doing: looping over your data, concatenating the data into numeric array is trivial when you have load-ed into a structure. Load them into separate variables and you will be writing the slowest, buggiest, and most obfuscated code in town.

Incidentally when you try to access variables dynamically all of MATLAB Editor's helpful tools (variable highlighting, code checking, error checking, f1 help, syntax help, etc, etc) do not work: why beginners prefer to use dynamic variable access which disables all of the MATLAB tools that help them to write good code is a mystery.

Have a look at this simple example (which I just copied from this question):

First we create some fake data:

>> A = 1:3;
>> save('file1.mat','A');
>> A = 4:6;
>> save('file2.mat','A');
>> A = 7:9;
>> save('file3.mat','A');

Now we get a list of those mat files, and read them in a loop:

S = dir('*.mat');
for k = 1:numel(S)
    tmp = load(S(k).name);
    S(k).A = tmp.A;
end

Now all the data is in structure S, and it is easy to access:

>> vertcat(S.A)
ans =
     1     2     3
     4     5     6
     7     8     9
>> S(3).A
ans =
     7     8     9

You should definitely read this: http://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html, and practice with the examples.

Of course you don't have to store the data in a structure, it is also trivial to store them in an array:

>> S = dir('*.mat');
>> M = NaN(numel(S),3);
>> for k = 1:numel(S), tmp = load(S(k).name); M(k,:) = tmp.A; end
>> M
M =
     1     2     3
     4     5     6
     7     8     9

[Kori Rogers]

Thanks for the brilliant answer. Could you please explain where .name comes from in

tmp = load(S(k).name) ?

[Stephen23]

"Could you please explain where .name comes from"

The function dir returns a structure array, which I allocated to the variable S. According to the dir documentation, one of its fields is 'name'. So the k-th element of that structure is S(k), and the name field of that element is S(k).name: it contains the name of the k-th file/folder found by dir.

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html