Handling data form structures

I always see people saying not to use eval so I decided to try and use a structures to load in my data. My code works but i feel like it can be done much easier. I have a 13x1 structure within the last field a 200x15 table containing data. Since my data originaly contians comma's instead of points I first need to covert it so matlab can work with it. Using dot indexing gives errors in many cases and strrep can't handle matrix. So my question is how to shorten my code and replace the table data, now in string form, with data in double form. Basicly I feel like there is no need for so many data type conversions. I would like to have my data in a table in my structure but of the type double. Now i can understand this may sound a bit fague so i attached my files. (Since i selected the wrong files I'm not allowed to upload because i reached the maximum amout of upload so i added the code below)

% Specifying the direction and file type
direction = 'C:\Users\tebra\OneDrive\Documents\School\Jaar 2\Experiments 3\Warmtewisselaar\Counter-flow';
type = '*.txt';

Data = dir(fullfile(direction,type)); % Getting all file names in the folder

% Loading all the data using the file names
for i=1:length(Data)
    name = Data(i).name;
    file = [direction '\' name];
    Data(i).data = readtable(file,"VariableNamingRule","preserve","Delimiter","tab");
end

% Looping through all datasets and obtaining the average
for i = 1:length(Data)
    Names = Data(i).data.Properties.VariableNames; %Getting the names of the variabels since the new table doesn't have variablenames
    UsedData1 = table2array(Data(i).data); % Convertingen the table2array since the original file uses comma's instead of points
   
    % Replacing all the comma's with points. Needs to be in a for loop because 'strrep' can't handle matrix 
    for ii = 1:width(UsedData1) 
        UsedData1(:,ii) = strrep(UsedData1(:,ii), ',', '.');
    end
    
    UsedData2 = cell2table(UsedData1); % Convert cell back to table so i can use dot indexing 
    Data(i).data = UsedData2; % Placing back the data into the structure
    Data(i).data.Properties.VariableNames = Names;

    % Getting all values
    % Since 'UsedData1' is a cell array, table 'UsedData2' contains the
    % data in string format. So to use the data convert to double
    FlowrateC = mean(str2double(Data(i).data.('dV2/dt (l/min)'))); %FlowrateC
    % ... other measured data
end

This is a small part of my text file maybe i load it in the wrong way.

G.U.N.T. Geraetebau GmbH
WL110



Time	dV1/dt (l/min)	dV2/dt (l/min)	T1 (°C)	T2 (°C)	T3 (°C)	T4 (°C)	T5 (°C)	T6 (°C)	dQ1/dt (kW)	dQ2/dt (kW)	dQm/dt (kW)	dTm (K)	km (kW/(m²*K))	Am (m²)
19/03/2024 15:56:08,72	2,22	2,25	58,7	54,9	51,2	22,2	25,6	29,8	-1,15	1,20	1,17	29,1	1,61	0,025
19/03/2024 15:56:09,22	2,22	2,27	58,7	54,9	51,2	22,0	25,6	29,7	-1,15	1,19	1,17	29,0	1,61	0,025
19/03/2024 15:56:09,72	2,24	2,28	58,7	54,9	51,2	21,9	25,5	29,6	-1,14	1,21	1,17	29,1	1,61	0,025

[Voss]

"strrep can't handle matrix"

Not true:

% a "cell matrix" if you will, i.e., a 2x3 cell array:
c = {'3,14' '2,78' '1,1'; '0,0' '93,7' '107,5'}

c = 2x3 cell array
    {'3,14'}    {'2,78'}    {'1,1'  }
    {'0,0' }    {'93,7'}    {'107,5'}

% strrep works as expected:
strrep(c,',','.')

ans = 2x3 cell array
    {'3.14'}    {'2.78'}    {'1.1'  }
    {'0.0' }    {'93.7'}    {'107.5'}

[Dirk te Brake]

I meant table, or does that work too?

[Voss]

Not for tables, no. You'd have to get the data out of the table first, or do convertvars to strrep in-place like in my answer.

[Stephen23]

Specify the decimal separator character to avoid messing about with text and STRREP and STR2DOUBLE:

T = readtable('test.txt', 'Delimiter','tab', 'DecimalSeparator',',', 'VariableNamingRule','preserve');
T.Time = datetime(T.Time, 'InputFormat','d/M/y H:m:s,SS')

T = 3x15 table
            Time            dV1/dt (l/min)    dV2/dt (l/min)    T1 (°C)    T2 (°C)    T3 (°C)    T4 (°C)    T5 (°C)    T6 (°C)    dQ1/dt (kW)    dQ2/dt (kW)    dQm/dt (kW)    dTm (K)    km (kW/(m²*K))    Am (m²)
    ____________________    ______________    ______________    _______    _______    _______    _______    _______    _______    ___________    ___________    ___________    _______    ______________    _______

    19-Mar-2024 15:56:08         2.22              2.25          58.7       54.9       51.2       22.2       25.6       29.8         -1.15           1.2           1.17         29.1           1.61          0.025 
    19-Mar-2024 15:56:09         2.22              2.27          58.7       54.9       51.2         22       25.6       29.7         -1.15          1.19           1.17           29           1.61          0.025 
    19-Mar-2024 15:56:09         2.24              2.28          58.7       54.9       51.2       21.9       25.5       29.6         -1.14          1.21           1.17         29.1           1.61          0.025 

[Dirk te Brake]

This works even better and is faster. I would accept this answer but you posted it as a comment instead of an answer.

Answer 1

[Voss]

% Specifying the direction and file type
direction = '.';
type = '*.txt';

Data = dir(fullfile(direction,type)); % Getting all file names in the folder

% Loading all the data using the file names
for i=1:length(Data)
    name = Data(i).name;
    % file = [direction '\' name];
    file = fullfile(direction,name);
    Data(i).data = readtable(file,"VariableNamingRule","preserve","Delimiter","tab");
end

% table variable conversion functions:
F1 = @(x)strrep(x,',','.');
F2 = @datetime;
F3 = @str2double;
    
% Looping through all datasets and obtaining the average
for i = 1:length(Data)
    
    T = Data(i).data; % for convenience of notation, store the table as T
    Nvars = width(T);
    
    T = convertvars(T,1:Nvars,F1); % replace ',' with '.' in all table variables
    T = convertvars(T,1,F2);       % convert variable 1 to datetime
    T = convertvars(T,2:Nvars,F3); % convert other variables to double
    
    Data(i).data = T; % store the new table back in Data
    
    % FlowrateC = mean(T.('dV2/dt (l/min)')) %FlowrateC
    FlowrateC = mean(T.(2)) % if you know 'dV2/dt (l/min)' is always the 2nd table, variable you can do this
    
end

FlowrateC = 2.2267

[Voss]

I gather that this line:

UsedData1(:,ii) = strrep(UsedData1(:,ii),"'","");

is intended to remove single-quotes, but there are no single-quotes in the file and none in the cell array UsedData1. (Single-quotes displayed in the command window at the start and end of a character array are just to indicate that something is a character array and are not actually part of the character array.)

Therefore, in my answer, I'm not doing that step, but if you need to do it, here's a way to do it and keep the data class the same (i.e., not convert to string like strrep(_,"'","") does):

C = {'ok''here','let''s go'}

C = 1x2 cell array
    {'ok'here'}    {'let's go'}

strrep(C,'''','')

ans = 1x2 cell array
    {'okhere'}    {'lets go'}

If you need to do this, you could combine this strrep call with the other one to replace commas:

F1 = @(x)strrep(strrep(x,',','.'),'''',''); % replace commas first, then remove single-quotes

[Dirk te Brake]

Thanks this is a lot cleaner, always forget to use anonymous functions. I still think cell arrays are not that user friendly, but maybe that's just me.

"UsedData1(:,ii) = strrep(UsedData1(:,ii),"'","");" I forgot to delete this part.

[Voss]

You're welcome!