How to identify and read similar files names in a folder and then create a matrix array with the data

Hello. I need to process hundreds of data copy in a folder from which the total data is separated into 3 with similar names as shown in the attached image (from here 5 output files will be identified). So, I will need to create a matrix array joining the data automatically in different files in a different folder .

Answer 1

[Mathieu NOE]

so an updated version of the code

to load data and headers of your files I used an older but still usefull readclm function (see end of file)

for the time being I am doing anything with the headers , it's up to you to decide if it's needed or not

hope it helps

clc
clearvars

path = pwd;  % me

S = dir(fullfile(path,'2021*.txt'));

%% export 
HeaderLines = 6; % header lines of data files
out_hne = [];
out_hnn = [];
out_hnz = [];
out_hne_filename = []; %  
out_hnn_filename = []; %  
out_hnz_filename = []; %  

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    F = fullfile(current_folder,current_filename);
     
    % case 1 :  "HNE"
    if contains(current_filename,'HNE')
        %data_hne = readmatrix(F,"NumHeaderLines",HeaderLines,"ExpectedNumVariables",1);
        [data_hne,header_hne] = readclm(F,1);
        out_hne = [out_hne;data_hne]; % data vertical concatenation
        out_hne_filename = [out_hne_filename;current_filename]; % debug 
        
     % case 2 :  "HNN"
   elseif contains(current_filename,'HNN')
%          data_hnn = readmatrix(F,"NumHeaderLines",HeaderLines,"ExpectedNumVariables",1);
         [data_hnn,header_hnn] = readclm(F,1);
       out_hnn = [out_hnn;data_hnn]; % data vertical concatenation   
        out_hnn_filename = [out_hnn_filename;current_filename]; % debug 

      % case 3 :  "HNZ"
    elseif contains(current_filename,'HNZ')
%         data_hnz = readmatrix(F,"NumHeaderLines",HeaderLines,"ExpectedNumVariables",1);
         [data_hnz,header_hnz] = readclm(F,1);
        out_hnz = [out_hnz;data_hnz]; % data vertical concatenation   
        out_hnz_filename = [out_hnz_filename;current_filename]; % debug 
       
    end

end

%% exports to /out folder

out_folder = [current_folder '/out'];

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));
disp(' HNE files processed according to this list :');
disp(out_hne_filename)
disp(' -------------------------------------------');

writematrix(out_hnn,fullfile(out_folder,'all_HNN.txt'));
disp(' HNN files processed according to this list :');
disp(out_hnn_filename)
disp(' -------------------------------------------');

writematrix(out_hnz,fullfile(out_folder,'all_HNZ.txt'));
disp(' HNZ files processed according to this list :');
disp(out_hnz_filename)
disp(' -------------------------------------------');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function  [outdata,head] = readclm(filename,nclm,skip,formt)

% READCLM Reads numerical data from a text file into a matrix.
%	Text file can begin with a header or comment block.
%	[DATA,HEAD] = READCLM(FILENAME,NCLM,SKIP,FORMAT)
%	Opens file FILENAME, skips first several lines specified
%	by SKIP number or beginning with comment '%'.
%	Then reads next several lines into a string matrix HEAD
%	until the first line with numerical data is encountered
%	(that is until first non-empty output of SSCANF).
%	Then reads the rest of the file into a numerical matrix
%	DATA in a format FORMAT with number of columns equal
%	to number of columns of the text file or specified by
%	number NCLM. If data does not match the size of the
%	matrix DATA, it is padded with NaN at the end.
%
%	READCLM(FILENAME) reads data from a text file FILENAME,
%	skipping only commented lines. It determines number of
%	columns by the length of the first data line and uses
%	the floating point format '%g';
%
%	READCLM uses FGETS to read the first lines and 	FSCANF
%	for reading data.

 % Defaults and  parameters ..............................
formt_dflt = '%g';  % Default format for fscanf
addn = nan;         % Number to fill the end if necessary

 % Handle input ..........................................
if nargin<1, error('  File name is undefined'); end
if nargin<4, formt = formt_dflt; end
if nargin<3, skip = 0; end
if nargin<2, nclm = 0; end
if isempty(nclm), nclm = 0; end
if isempty(skip), skip = 0; end

 % Open file ............................
[fid,msg] = fopen(filename);
if fid<0, disp(msg), return, end

 % Find header and first  data line ......................
is_head = 1;
jl = 0;
head = ' ';
while is_head  % Add lines to header.....
  s = fgets(fid);           % Get next line
  jl = jl+1;
  is_skip = jl<=skip;
  is_skip = jl<=skip | s(1)=='%';
  out1 = sscanf(s,formt);   % Try to read this line
   % If unreadable by SSCANF or skip, add to header
  is_head = isempty(out1) | is_skip;
  if is_head & ~is_skip
    head = str2mat(head,s(1:length(s)-1)); end
end
head = head(2:size(head,1),:);

 % Determine number of columns if not specified
out1 = out1(:)';
l1 = length(out1);
if ~nclm, nclm = l1; end

 % Read the rest of the file ..............................
if l1~=nclm  % First line format is different from ncolumns

  outdata = fscanf(fid,formt);
  lout = length(outdata)+l1;
  ncu = ceil(lout/nclm);
  lz = nclm*ncu-lout;
  outdata = [out1'; outdata(:); ones(lz,1)*addn];
  outdata = reshape(outdata,nclm,ncu)';

else              % Regular case

  outdata = fscanf(fid,formt,[nclm inf]);
  outdata = [out1; outdata'];  % Add the first line

end

fclose (fid);     % Close file ..........
end

[Jorge Luis Paredes Estacio]

Thank you very much for the information provided. I attached some files as example.

[Jorge Luis Paredes Estacio]

Only the values after '# Unidades: m/seg/seg' is the information that will be collected and put all together in columns and the number of sample must coincide. If not, the maximum number of samples prevails and the other two files will be filled with zeros to match the quantity of data for each 3 files.

[Jorge Luis Paredes Estacio]

a new file with 3 separate columns with the same amount of data for each three files that have similar name

[Mathieu NOE]

the code has been changed so the data are horizontally concatenated with zero padding if needed

the resulting arry have same number of rows as the longest input file

clc
clearvars

path = pwd;  % me

S = dir(fullfile(path,'2021*.txt'));

%% export 
HeaderLines = 6; % header lines of data files
out_hne = [];
out_hnn = [];
out_hnz = [];
out_hne_filename = []; %  
out_hnn_filename = []; %  
out_hnz_filename = []; %  

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    F = fullfile(current_folder,current_filename);
     
    % case 1 :  "HNE"
    if contains(current_filename,'HNE')
        [data_hne,header_hne] = readclm(F,1);
        out_hne = horz_concat_uneven(out_hne,data_hne); % data horizontal concatenation
        out_hne_filename = [out_hne_filename;current_filename]; % debug 
        
     % case 2 :  "HNN"
   elseif contains(current_filename,'HNN')
         [data_hnn,header_hnn] = readclm(F,1);
        out_hnn = horz_concat_uneven(out_hnn,data_hnn); % data horizontal concatenation
        out_hnn_filename = [out_hnn_filename;current_filename]; % debug 

      % case 3 :  "HNZ"
    elseif contains(current_filename,'HNZ')
         [data_hnz,header_hnz] = readclm(F,1);
        out_hnz = horz_concat_uneven(out_hnz,data_hnz); % data horizontal concatenation
        out_hnz_filename = [out_hnz_filename;current_filename]; % debug 
       
    end

end

%% exports to /out folder

out_folder = [current_folder '/out'];

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));
disp(' HNE files processed according to this list :');
disp(out_hne_filename)
disp(' -------------------------------------------');

writematrix(out_hnn,fullfile(out_folder,'all_HNN.txt'));
disp(' HNN files processed according to this list :');
disp(out_hnn_filename)
disp(' -------------------------------------------');

writematrix(out_hnz,fullfile(out_folder,'all_HNZ.txt'));
disp(' HNZ files processed according to this list :');
disp(out_hnz_filename)
disp(' -------------------------------------------');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function out = horz_concat_uneven(array1,array2)

     % check arrays dimensions
     [m1,n1] = size(array1);
     [m2,n2] = size(array2);
     mm = max(m1,m2);
     
     % fill out with data
     out = zeros(mm,n1+n2);
     out(1:m1,1:n1) = array1;
     out(1:m2,n1+(1:n2)) = array2;
     
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function  [outdata,head] = readclm(filename,nclm,skip,formt)

% READCLM Reads numerical data from a text file into a matrix.
%	Text file can begin with a header or comment block.
%	[DATA,HEAD] = READCLM(FILENAME,NCLM,SKIP,FORMAT)
%	Opens file FILENAME, skips first several lines specified
%	by SKIP number or beginning with comment '%'.
%	Then reads next several lines into a string matrix HEAD
%	until the first line with numerical data is encountered
%	(that is until first non-empty output of SSCANF).
%	Then reads the rest of the file into a numerical matrix
%	DATA in a format FORMAT with number of columns equal
%	to number of columns of the text file or specified by
%	number NCLM. If data does not match the size of the
%	matrix DATA, it is padded with NaN at the end.
%
%	READCLM(FILENAME) reads data from a text file FILENAME,
%	skipping only commented lines. It determines number of
%	columns by the length of the first data line and uses
%	the floating point format '%g';
%
%	READCLM uses FGETS to read the first lines and 	FSCANF
%	for reading data.

 % Defaults and  parameters ..............................
formt_dflt = '%g';  % Default format for fscanf
addn = nan;         % Number to fill the end if necessary

 % Handle input ..........................................
if nargin<1, error('  File name is undefined'); end
if nargin<4, formt = formt_dflt; end
if nargin<3, skip = 0; end
if nargin<2, nclm = 0; end
if isempty(nclm), nclm = 0; end
if isempty(skip), skip = 0; end

 % Open file ............................
[fid,msg] = fopen(filename);
if fid<0, disp(msg), return, end

 % Find header and first  data line ......................
is_head = 1;
jl = 0;
head = ' ';
while is_head  % Add lines to header.....
  s = fgets(fid);           % Get next line
  jl = jl+1;
  is_skip = jl<=skip;
  is_skip = jl<=skip | s(1)=='%';
  out1 = sscanf(s,formt);   % Try to read this line
   % If unreadable by SSCANF or skip, add to header
  is_head = isempty(out1) | is_skip;
  if is_head & ~is_skip
    head = str2mat(head,s(1:length(s)-1)); end
end
head = head(2:size(head,1),:);

 % Determine number of columns if not specified
out1 = out1(:)';
l1 = length(out1);
if ~nclm, nclm = l1; end

 % Read the rest of the file ..............................
if l1~=nclm  % First line format is different from ncolumns

  outdata = fscanf(fid,formt);
  lout = length(outdata)+l1;
  ncu = ceil(lout/nclm);
  lz = nclm*ncu-lout;
  outdata = [out1'; outdata(:); ones(lz,1)*addn];
  outdata = reshape(outdata,nclm,ncu)';

else              % Regular case

  outdata = fscanf(fid,formt,[nclm inf]);
  outdata = [out1; outdata'];  % Add the first line

end

fclose (fid);     % Close file ..........
end

[Jorge Luis Paredes Estacio]

THANK YOU VERY MUCH, I RUN THE SCRIPT AND APPEARS THIS MESSAGE

Error in Read_data_chile_joining (line 39)

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));

THANK YOU FOR YOUR RESPSONSE

[Mathieu NOE]

hello

can you share the entire error message ?

which matlab release are you using ?

maybe writematrix was not yet in your version (then it shuld appears in the error message that writematrix is unknown)

I'll work on an alternative in the mean time

[Mathieu NOE]

hello again

in this version of the code I replaced writematrix with the older dlmwrite

see if it works on your side

clc
clearvars

path = pwd;  % me

S = dir(fullfile(path,'2021*.txt'));

%% export 
HeaderLines = 6; % header lines of data files
out_hne = [];
out_hnn = [];
out_hnz = [];
out_hne_filename = []; %  
out_hnn_filename = []; %  
out_hnz_filename = []; %  

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    F = fullfile(current_folder,current_filename);
     
    % case 1 :  "HNE"
    if contains(current_filename,'HNE')
        [data_hne,header_hne] = readclm(F,1);
        out_hne = horz_concat_uneven(out_hne,data_hne); % data horizontal concatenation
        out_hne_filename = [out_hne_filename;current_filename]; % debug 
        
     % case 2 :  "HNN"
   elseif contains(current_filename,'HNN')
         [data_hnn,header_hnn] = readclm(F,1);
        out_hnn = horz_concat_uneven(out_hnn,data_hnn); % data horizontal concatenation
        out_hnn_filename = [out_hnn_filename;current_filename]; % debug 

      % case 3 :  "HNZ"
    elseif contains(current_filename,'HNZ')
         [data_hnz,header_hnz] = readclm(F,1);
        out_hnz = horz_concat_uneven(out_hnz,data_hnz); % data horizontal concatenation
        out_hnz_filename = [out_hnz_filename;current_filename]; % debug 
       
    end

end

%% exports to /out folder

out_folder = [current_folder '/out'];

% writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));
dlmwrite(fullfile(out_folder,'all_HNE.txt'),out_hne);
disp(' HNE files processed according to this list :');
disp(out_hne_filename)
disp(' -------------------------------------------');

%writematrix(out_hnn,fullfile(out_folder,'all_HNN.txt'));
dlmwrite(fullfile(out_folder,'all_HNN.txt'),out_hnn);
disp(' HNN files processed according to this list :');
disp(out_hnn_filename)
disp(' -------------------------------------------');

% writematrix(out_hnz,fullfile(out_folder,'all_HNZ.txt'));
dlmwrite(fullfile(out_folder,'all_HNZ.txt'),out_hnz);
disp(' HNZ files processed according to this list :');
disp(out_hnz_filename)
disp(' -------------------------------------------');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function out = horz_concat_uneven(array1,array2)

     % check arrays dimensions
     [m1,n1] = size(array1);
     [m2,n2] = size(array2);
     mm = max(m1,m2);
     
     % fill out with data
     out = zeros(mm,n1+n2);
     out(1:m1,1:n1) = array1;
     out(1:m2,n1+(1:n2)) = array2;
     
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function  [outdata,head] = readclm(filename,nclm,skip,formt)

% READCLM Reads numerical data from a text file into a matrix.
%	Text file can begin with a header or comment block.
%	[DATA,HEAD] = READCLM(FILENAME,NCLM,SKIP,FORMAT)
%	Opens file FILENAME, skips first several lines specified
%	by SKIP number or beginning with comment '%'.
%	Then reads next several lines into a string matrix HEAD
%	until the first line with numerical data is encountered
%	(that is until first non-empty output of SSCANF).
%	Then reads the rest of the file into a numerical matrix
%	DATA in a format FORMAT with number of columns equal
%	to number of columns of the text file or specified by
%	number NCLM. If data does not match the size of the
%	matrix DATA, it is padded with NaN at the end.
%
%	READCLM(FILENAME) reads data from a text file FILENAME,
%	skipping only commented lines. It determines number of
%	columns by the length of the first data line and uses
%	the floating point format '%g';
%
%	READCLM uses FGETS to read the first lines and 	FSCANF
%	for reading data.

 % Defaults and  parameters ..............................
formt_dflt = '%g';  % Default format for fscanf
addn = nan;         % Number to fill the end if necessary

 % Handle input ..........................................
if nargin<1, error('  File name is undefined'); end
if nargin<4, formt = formt_dflt; end
if nargin<3, skip = 0; end
if nargin<2, nclm = 0; end
if isempty(nclm), nclm = 0; end
if isempty(skip), skip = 0; end

 % Open file ............................
[fid,msg] = fopen(filename);
if fid<0, disp(msg), return, end

 % Find header and first  data line ......................
is_head = 1;
jl = 0;
head = ' ';
while is_head  % Add lines to header.....
  s = fgets(fid);           % Get next line
  jl = jl+1;
  is_skip = jl<=skip;
  is_skip = jl<=skip | s(1)=='%';
  out1 = sscanf(s,formt);   % Try to read this line
   % If unreadable by SSCANF or skip, add to header
  is_head = isempty(out1) | is_skip;
  if is_head & ~is_skip
    head = str2mat(head,s(1:length(s)-1)); end
end
head = head(2:size(head,1),:);

 % Determine number of columns if not specified
out1 = out1(:)';
l1 = length(out1);
if ~nclm, nclm = l1; end

 % Read the rest of the file ..............................
if l1~=nclm  % First line format is different from ncolumns

  outdata = fscanf(fid,formt);
  lout = length(outdata)+l1;
  ncu = ceil(lout/nclm);
  lz = nclm*ncu-lout;
  outdata = [out1'; outdata(:); ones(lz,1)*addn];
  outdata = reshape(outdata,nclm,ncu)';

else              % Regular case

  outdata = fscanf(fid,formt,[nclm inf]);
  outdata = [out1; outdata'];  % Add the first line

end

fclose (fid);     % Close file ..........
end

[Jorge Luis Paredes Estacio]

I am using Matlab R2020a. This was the whole error message:

Error using writematrix (line 156)

Unable to open file 'E:\Civil Egineering PhD\Processing data matlab\matlab code\MOTION RECORDS CSN CHILE MEJORADO\out\all_HNE.txt' for writing:

No such file or directory

Error in Read_data_chile_joining (line 39)

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));

I am going to try this new script. Thank you very much.

[Jorge Luis Paredes Estacio]

The error message is the same:

Error using dlmwrite (line 116)

Cannot open file E:\Civil Egineering PhD\Processing data matlab\matlab code\MOTION RECORDS CSN CHILE MEJORADO\out\all_HNE.txt.

Error in Read_data_chile_joining (line 40)

dlmwrite(fullfile(out_folder,'all_HNE.txt'),out_hne);

[Mathieu NOE]

ok

I think I understand now ...

forgot to tell you to create the subfolder (...\out) before you run the script !

see those lines in the code :

%% exports to /out folder
out_folder = [current_folder '\out'];

(remember you asked the output to be in a different folder : So, I will need to create a matrix array joining the data automatically in different files in a different folder .)

if the subfolder does not exist then matlab cannot store the results in a non existent folder

either create it with your windows explorer , or you can do it with matlab too :

Make new folder - MATLAB mkdir - MathWorks France

[Mathieu NOE]

see also the error message you get :

No such file or directory

[Jorge Luis Paredes Estacio]

It run this time. Thank you very much. However, the files should be joined for each three files. I mean. For example, the file 20211104-041752-C16O has three components: HNE, HNN, and HNZ. The same applies for the whole remaning files in the folder that can be hundreds. So, for this example (e.g. 20211104-041752-C16O), one file is created where the first, second and third column are placed the components of HNE, HNN and HNZ. Herein, the maximum number of data sample of one of the thre components prevails to maintain the same amount of samples in each colum. That means that any of the rest two columns is filled with zeros to maintain the same sample number. If we have 15 files in the folder as initial run, 5 files will be created in the "OUT" folder. Each of the 5 files will have 3 columns with the same amount of samples for each file (this amount of samples can vary among the 5 files generated for example one can have 2500 for the three colums, the other 3650 for the three columns, and so on). Thank you very much for the support.

[Jorge Luis Paredes Estacio]

By the way, the files created in the "out" folder should maintain the name without the names of the components. For example 20211104-041752-C16O, 20211104-041752-GO04, and so on.

[Mathieu NOE]

ok i think I start to understand...

check this new code :

clc
clearvars

path = pwd;  % me

S = dir(fullfile(path,'2021*.txt'));

%% create  list of unique filenames like 20211104-041752-C16O 
% then load and save the three components: HNE, HNN, and HNZ in one out
% file

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    short_filename{k} = current_filename(:,1:length(current_filename)-8);
    end

% list of unique files : 
filename_out = (unique(short_filename))';  % like {'20211104-041733-C10O'}    {'20211104-041733-C20O'}    {'20211104-041740-C33O'}

%% main loop
for k = 1:numel(filename_out)
    current_folder = S(1).folder; % same folder indeed
    current_filename  = filename_out(k); % from list above (unique names)
    F = fullfile(current_folder,current_filename);
     
    % load  "HNE"
        F = fullfile(current_folder,[current_filename{:} '-HNE.txt']);
        [data_hne,header_hne] = readclm(F,1);
    % load  "HNN"
        F = fullfile(current_folder,[current_filename{:} '-HNN.txt']);
        [data_hnn,header_hnn] = readclm(F,1);
    % load  "HNZ"
        F = fullfile(current_folder,[current_filename{:} '-HNZ.txt']);
        [data_hnz,header_hnz] = readclm(F,1);
        
        out = [];
        out = horz_concat_uneven(data_hne,data_hnn); % data horizontal concatenation
        out = horz_concat_uneven(out,data_hnz); % data horizontal concatenation
        
        %% exports to /out folder
        out_folder = [current_folder '\out'];

        disp(' -------------------------------------------');
        writematrix(out,fullfile(out_folder,[current_filename{:} '.txt']));
        disp([ current_filename{:} ' files processed']);
        disp(' -------------------------------------------');
        

end


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function out = horz_concat_uneven(array1,array2)

     % check arrays dimensions
     [m1,n1] = size(array1);
     [m2,n2] = size(array2);
     mm = max(m1,m2);
     
     % fill out with data
     out = zeros(mm,n1+n2);
     out(1:m1,1:n1) = array1;
     out(1:m2,n1+(1:n2)) = array2;
     
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function  [outdata,head] = readclm(filename,nclm,skip,formt)

% READCLM Reads numerical data from a text file into a matrix.
%	Text file can begin with a header or comment block.
%	[DATA,HEAD] = READCLM(FILENAME,NCLM,SKIP,FORMAT)
%	Opens file FILENAME, skips first several lines specified
%	by SKIP number or beginning with comment '%'.
%	Then reads next several lines into a string matrix HEAD
%	until the first line with numerical data is encountered
%	(that is until first non-empty output of SSCANF).
%	Then reads the rest of the file into a numerical matrix
%	DATA in a format FORMAT with number of columns equal
%	to number of columns of the text file or specified by
%	number NCLM. If data does not match the size of the
%	matrix DATA, it is padded with NaN at the end.
%
%	READCLM(FILENAME) reads data from a text file FILENAME,
%	skipping only commented lines. It determines number of
%	columns by the length of the first data line and uses
%	the floating point format '%g';
%
%	READCLM uses FGETS to read the first lines and 	FSCANF
%	for reading data.

 % Defaults and  parameters ..............................
formt_dflt = '%g';  % Default format for fscanf
addn = nan;         % Number to fill the end if necessary

 % Handle input ..........................................
if nargin<1, error('  File name is undefined'); end
if nargin<4, formt = formt_dflt; end
if nargin<3, skip = 0; end
if nargin<2, nclm = 0; end
if isempty(nclm), nclm = 0; end
if isempty(skip), skip = 0; end

 % Open file ............................
[fid,msg] = fopen(filename);
if fid<0, disp(msg), return, end

 % Find header and first  data line ......................
is_head = 1;
jl = 0;
head = ' ';
while is_head  % Add lines to header.....
  s = fgets(fid);           % Get next line
  jl = jl+1;
  is_skip = jl<=skip;
  is_skip = jl<=skip | s(1)=='%';
  out1 = sscanf(s,formt);   % Try to read this line
   % If unreadable by SSCANF or skip, add to header
  is_head = isempty(out1) | is_skip;
  if is_head & ~is_skip
    head = str2mat(head,s(1:length(s)-1)); end
end
head = head(2:size(head,1),:);

 % Determine number of columns if not specified
out1 = out1(:)';
l1 = length(out1);
if ~nclm, nclm = l1; end

 % Read the rest of the file ..............................
if l1~=nclm  % First line format is different from ncolumns

  outdata = fscanf(fid,formt);
  lout = length(outdata)+l1;
  ncu = ceil(lout/nclm);
  lz = nclm*ncu-lout;
  outdata = [out1'; outdata(:); ones(lz,1)*addn];
  outdata = reshape(outdata,nclm,ncu)';

else              % Regular case

  outdata = fscanf(fid,formt,[nclm inf]);
  outdata = [out1; outdata'];  % Add the first line

end

fclose (fid);     % Close file ..........
end

[Jorge Luis Paredes Estacio]

That is perfect. Thank you very much. I was wondering if three rows can be added to each file joined (for example the 5 files generated in the OUT folder from the 15 files). The first row will be X, Y and Z, the second row the number of data samples unified for the three components (e.g. 2500) and the third row will have the value extracted from the original file that will be the same for the 3 components. For example: each file contains this information:

# Tasa de muestreo: 200.0 muestras/seg (here the value that matters is 200)

So the three first rows after the joined data will be:

1st row: X Y Z (fixed components)

2nd row: 2500 2500 2500 (this is the number of sample unified for the three components)

3rd row: 200 200 200 (this is the information extracted from the three joined files that is the same. This can be found in 200211104-041733-C10O-HNE or 200211104-041733-C10O-HNN or 200211104-041733-C10O-HNZ)

4th row until the end of the unified samples numbers: the joined data will follow for the 3 components.

THANK YOU VERY MUCH FOR ALL THIS COMPLEX CODE!!!!.

[Jorge Luis Paredes Estacio]

It worth mentioning that the # Tasa de muestreo: 200.0 muestras/seg can vary for each joined file. But it will be the same for the three components. For example: the joined file 20211104-041733-C10O may have a different value from the joined file 20211104-041733-C20O and so on.

[Mathieu NOE]

hello again

ok, i did some modifications

try this new code :

clc
clearvars

path = pwd;  %

S = dir(fullfile(path,'2021*.txt'));

%% create  list of unique filenames like 20211104-041752-C16O 
% then load and save the three components: HNE, HNN, and HNZ in one out file

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    short_filename{k} = current_filename(:,1:length(current_filename)-8);
    end

% list of unique files : 
filename_out = (unique(short_filename))';  % like {'20211104-041733-C10O'}    {'20211104-041733-C20O'}    {'20211104-041740-C33O'}

%% main loop
for k = 1:numel(filename_out)
    current_folder = S(1).folder; % same folder indeed
    current_filename  = filename_out(k); % from list above (unique names)
    F = fullfile(current_folder,current_filename);
     
    % load  "HNE"
        F = fullfile(current_folder,[current_filename{:} '-HNE.txt']);
        [data_hne,header_hne] = readclm(F,1);
        fs_hne = regexp(header_hne(2,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % sampling rate
        samples_hne = regexp(header_hne(3,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % total samples
        
    % load  "HNN"
        F = fullfile(current_folder,[current_filename{:} '-HNN.txt']);
        [data_hnn,header_hnn] = readclm(F,1);
         fs_hnn = regexp(header_hnn(2,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % sampling rate
         samples_hnn = regexp(header_hnn(3,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % total samples
      
    % load  "HNZ"
        F = fullfile(current_folder,[current_filename{:} '-HNZ.txt']);
        [data_hnz,header_hnz] = readclm(F,1);
        fs_hnz = regexp(header_hnz(2,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % sampling rate
        samples_hnz = regexp(header_hnz(3,:),'-?\d+\.?\d*|-?\d*\.?\d+','match'); % total samples
       
        % same amount of samples for one file / 3 components (HNE/HNN/HNZ)
        % so horizontal concatenation made simple : (no need to account for
        % size mismatch:
        out = [data_hne data_hnn data_hnz];

        %% exports to /out folder
        out_folder = [current_folder '\out'];
        file_and_folder = fullfile(out_folder,[current_filename{:} '.txt']);

        disp(' -------------------------------------------');
        writecell({'X','Y','Z'},file_and_folder);
        writecell([samples_hne samples_hnn samples_hnz],file_and_folder,"WriteMode","append");
        writecell([fs_hne fs_hnn fs_hnz],file_and_folder,"WriteMode","append");
        writematrix(out,file_and_folder,"WriteMode","append");
        disp([ current_filename{:} ' files processed']);
        disp(' -------------------------------------------');
        

end


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function  [outdata,head] = readclm(filename,nclm,skip,formt)

% READCLM Reads numerical data from a text file into a matrix.
%	Text file can begin with a header or comment block.
%	[DATA,HEAD] = READCLM(FILENAME,NCLM,SKIP,FORMAT)
%	Opens file FILENAME, skips first several lines specified
%	by SKIP number or beginning with comment '%'.
%	Then reads next several lines into a string matrix HEAD
%	until the first line with numerical data is encountered
%	(that is until first non-empty output of SSCANF).
%	Then reads the rest of the file into a numerical matrix
%	DATA in a format FORMAT with number of columns equal
%	to number of columns of the text file or specified by
%	number NCLM. If data does not match the size of the
%	matrix DATA, it is padded with NaN at the end.
%
%	READCLM(FILENAME) reads data from a text file FILENAME,
%	skipping only commented lines. It determines number of
%	columns by the length of the first data line and uses
%	the floating point format '%g';
%
%	READCLM uses FGETS to read the first lines and 	FSCANF
%	for reading data.

 % Defaults and  parameters ..............................
formt_dflt = '%g';  % Default format for fscanf
addn = nan;         % Number to fill the end if necessary

 % Handle input ..........................................
if nargin<1, error('  File name is undefined'); end
if nargin<4, formt = formt_dflt; end
if nargin<3, skip = 0; end
if nargin<2, nclm = 0; end
if isempty(nclm), nclm = 0; end
if isempty(skip), skip = 0; end

 % Open file ............................
[fid,msg] = fopen(filename);
if fid<0, disp(msg), return, end

 % Find header and first  data line ......................
is_head = 1;
jl = 0;
head = ' ';
while is_head  % Add lines to header.....
  s = fgets(fid);           % Get next line
  jl = jl+1;
  is_skip = jl<=skip;
  is_skip = jl<=skip | s(1)=='%';
  out1 = sscanf(s,formt);   % Try to read this line
   % If unreadable by SSCANF or skip, add to header
  is_head = isempty(out1) | is_skip;
  if is_head & ~is_skip
    head = str2mat(head,s(1:length(s)-1)); end
end
head = head(2:size(head,1),:);

 % Determine number of columns if not specified
out1 = out1(:)';
l1 = length(out1);
if ~nclm, nclm = l1; end

 % Read the rest of the file ..............................
if l1~=nclm  % First line format is different from ncolumns

  outdata = fscanf(fid,formt);
  lout = length(outdata)+l1;
  ncu = ceil(lout/nclm);
  lz = nclm*ncu-lout;
  outdata = [out1'; outdata(:); ones(lz,1)*addn];
  outdata = reshape(outdata,nclm,ncu)';

else              % Regular case

  outdata = fscanf(fid,formt,[nclm inf]);
  outdata = [out1; outdata'];  % Add the first line

end

fclose (fid);     % Close file ..........
end

[Jorge Luis Paredes Estacio]

Thank you so much. There is a error when runnin though.

Error using horzcat

Dimensions of arrays being concatenated are not consistent.

Error in Read_data_chile_joining (line 41)

out = [data_hne data_hnn data_hnz];

[Mathieu NOE]

haha ?...

did that occur with the provided txt files ? I did not have this issue

so what are the files that created this error ? normally all hne / hnn / hnz files have same length for one condition

can you share the files ?

[Mathieu NOE]

you can zip hne / hnn / hnz files for each one condition

its saves space and time to download...

[Jorge Luis Paredes Estacio]

Thank you very much. I have this script to read some files in a specific folder name "MOTION RECORDS RENADIC_V1" where thre will be many data but for this example I attached the 4 files in it. Could you please help me out to add two additional conditions when reading the data with two additional specific details in it. The function name is "import_from_chile_v1 and the additional conditions from line 59 to 54 (highlighted in the script down there) is to include that 'L' will be indexing (component) =1 and 'T' will be indexing (component) =2. This will solve the issue of reading the other files that have a different component of "EW" and "NS". THANK YOU FOR YOUR SUPPORT!.

Script:

close all

clear

clc

addpath(pwd)

cd('MOTION RECORDS RENADIC_V1')

names = dir;

names(1:2)=[];

wb = waitbar(0,'Please wait...');

for i = 1:length(names)

waitbar(i/length(names),wb,['Record ', num2str(i), ' of ', num2str(length(names))])

filename = names(i).name;

[dt, Output] = import_from_chile_v1(filename);

STDIR = ["X", "Y", "Z"]; %STATION MOTION DIRECTIONALITY

%%%%Reading acceleration data%%%%%%

Length_Signal = max(size(Output));

Acc=[STDIR; Length_Signal Length_Signal Length_Signal; 1/dt 1/dt 1/dt; Output(:,1) Output(:,2) Output(:,3)];

cd ..

cd('SORTED RECORDS')

writematrix(Acc,[filename(1:end-3),'.txt'],'Delimiter','tab');

cd ..

cd('MOTION RECORDS RENADIC_V1')

end

close(wb)

%% function name: import_from_chile_v1

function [dt, Output] = import_from_chile_v1(filename)

stopper_line = '/& ----------';

len_stop_line = length(stopper_line);

channel_line = 'CHAN ';

len_chan_line = length(channel_line);

fid = fopen(filename,'r');

tline = fgetl(fid);

i=1;

tmp_1 = [];

tmp_2 = [];

tmp_3 = [];

component = 1;

line_tmp = 0;

indexing = zeros(1,3);

while ischar(tline)

i=i+1;

tline = fgetl(fid);

len_tline = length(tline);

if len_tline>len_chan_line

if strcmp(tline(1:len_chan_line),channel_line)

if strcmp(tline(10),'E')

indexing(component) = 1;

elseif strcmp(tline(10),'N')

indexing(component) = 2;

elseif strcmp(tline(10),'V')

indexing(component) = 3;

end

end

end

if i>27 & component == 1

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if isempty(tmp_1)

tmp_1 = str2num(convertCharsToStrings(strip(tline)));

else

tmp_1 = [tmp_1; str2num(convertCharsToStrings(strip(tline)))];

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

elseif i>line_tmp+27 & component == 2

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if isempty(tmp_2)

tmp_2 = str2num(convertCharsToStrings(strip(tline)));

else

tmp_2 = [tmp_2; str2num(convertCharsToStrings(strip(tline)))];

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

elseif i>line_tmp+27 & component == 3

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if isempty(tmp_3)

tmp_3 = str2num(convertCharsToStrings(strip(tline)));

else

tmp_3 = [tmp_3; str2num(convertCharsToStrings(strip(tline)))];

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

end

end

dt = tmp_1(1,3)-tmp_1(1,1);

tmp_1 = tmp_1'; tmp_1 = tmp_1(:);

tmp_2 = tmp_2'; tmp_2 = tmp_2(:);

tmp_3 = tmp_3'; tmp_3 = tmp_3(:);

tmp = [tmp_1(2:2:end)*98.0665, tmp_2(2:2:end)*98.0665, tmp_3(2:2:end)*98.0665];

Output =[tmp(:,indexing==1) tmp(:,indexing==2) tmp(:,indexing==3)];

end

[Mathieu NOE]

hello

ok, had to take some time to figure out this new challenge :)

I had some issues running the original code so I modified mostly the sections where you create the tmp_1 to tmp_3 arrays (that took some time...)

I added the L and T conditions as you wished (that was fairly easy)

so as far as I have understood , the goal is to get 3 channels data organized as follows :

1st column (X) : E or L

2nd column (Y) : N or T

3rd column (Z) : V

this is the upadted code. there are section still commented just to avoid the forth and back directory swap with cd - I leave it to put it back in operation !

the code has been tested with the 4 files you provided

close all
clear
clc

% addpath(pwd)
% cd('MOTION RECORDS RENADIC_V1')
names = dir;

names = dir('*.txt');
% names(1:2)=[];

wb = waitbar(0,'Please wait...');
for i = 1:length(names)   
    
waitbar(i/length(names),wb,['Record ', num2str(i), ' of ',  num2str(length(names))])    ;
filename = names(i).name; 
    
[dt, time, Output, indexing] = import_from_chile_v2(filename);

     %%  we want 
     % 1st column (X) : E or L
     % 2nd column (Y) : N or T
     % 3rd column (Z) : V
     % so we need to reorganize columns according to indexing : 
     Output = Output(:,indexing);
    
% modified here for a quick validation of the export 
STDIR = ["X", "Y", "Z"];    %STATION MOTION DIRECTIONALITY
%%%%Reading acceleration data%%%%%%
Length_Signal = max(size(Output));
Acc=[STDIR; Length_Signal Length_Signal Length_Signal; 1/dt 1/dt 1/dt; Output];
writematrix(Acc,[filename(1:end-4),'.dat'],'Delimiter','tab');

% cd ..
% cd('SORTED RECORDS')
% writematrix(Acc,[filename(1:end-4),'.txt'],'Delimiter','tab');
% cd ..
% cd('MOTION RECORDS RENADIC_V1')

end

close(wb)

%% function name: import_from_chile_v2

function [dt, time, Output, indexing] = import_from_chile_v2(filename)

stopper_line  = '/&  ----------';
len_stop_line = length(stopper_line);

channel_line = 'CHAN  ';
len_chan_line = length(channel_line);

fid = fopen(filename,'r');

tline = fgetl(fid);
i=1;
tmp_1 = [];
tmp_2 = [];
tmp_3 = [];
component = 1;
line_tmp = 0;
indexing = zeros(1,3);

while ischar(tline)
    
    i=i+1;
    tline = fgetl(fid);
    len_tline = length(tline);
    
    if len_tline>len_chan_line
        if strcmp(tline(1:len_chan_line),channel_line)
            if strcmp(tline(10),'E') || strcmp(tline(10),'L')
                indexing(component) = 1;
            elseif strcmp(tline(10),'N') || strcmp(tline(10),'T')
                indexing(component) = 2;
            elseif strcmp(tline(10),'V')
                indexing(component) = 3;
            end
        end
    end
    
    if i>27 && component == 1
        
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        if len_tline > len_stop_line
            if strcmp(tline(1:len_stop_line),stopper_line)
                line_tmp = i;
                component = component + 1;
            end
        end
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        tmp_1 = [tmp_1; reshape_2cols(str2num(tline))];
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        
    elseif i>line_tmp+27 && component == 2
        
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        if len_tline > len_stop_line
            if strcmp(tline(1:len_stop_line),stopper_line)
                line_tmp = i;
                component = component + 1;
            end
        end
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
       tmp_2 = [tmp_2; reshape_2cols(str2num(tline))];
       %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        
    elseif i>line_tmp+27  && component == 3
        
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        if len_tline > len_stop_line
            if strcmp(tline(1:len_stop_line),stopper_line)
                line_tmp = i;
                component = component + 1;
            end
        end
        %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
        tmp_3 = [tmp_3; reshape_2cols(str2num(tline))];
       %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

    end
end

time = tmp_1(:,1);
dt = mean(diff(time));
accel_factor = 98.0665; % conversion from G/10 to m/s²
tmp = accel_factor*[tmp_1(:,2) tmp_2(:,2) tmp_3(:,2)]; % vertical concatenation
Output =[tmp(:,indexing==1) tmp(:,indexing==2) tmp(:,indexing==3)];

end

%%%%%%%%% functions %%%%%%%%

function out = reshape_2cols(in)
% reshape 2N columns array in 2 columns array %

    a = in(:,1:2:end); % Select the odd numbered columns
    b = in(:,2:2:end); % Select the even numbered columns
    out = [a(:) b(:)]; % Append them

end

[Jorge Luis Paredes Estacio]

That was very lovely. This is anoher challenge that was adapted to a new data using the same script but in a more complex way of distribution of data. This reading also considers the following scenarios:

1st column (X) : EW or L

2nd column (Y) : NS or T

3rd column (Z) : V

However, the L and T reading is not working. Could you please check the matlab code. I also attached the examples files for reading that should be located in a folder named "MOTION RECORDS RENADIC_V2"

Script:

close all

clear

clc

addpath(pwd)

cd('MOTION RECORDS RENADIC_V2')

names = dir;

names = dir('*.v2');

% names(1:2)=[];

wb = waitbar(0,'Please wait...');

for i = 1:length(names)

waitbar(i/length(names),wb,['Record ', num2str(i), ' of ', num2str(length(names))]) ;

filename = names(i).name;

[dt, time, Output, indexing] = import_from_chile_v2_corrected(filename);

%% we want

% 1st column (X) : EW or T

% 2nd column (Y) : NS or L

% 3rd column (Z) : V

% modified here for a quick validation of the export

STDIR = ["X", "Y", "Z"]; %STATION MOTION DIRECTIONALITY

%%%%Reading acceleration data%%%%%%

Length_Signal = max(size(Output));

Acc=[STDIR; Length_Signal Length_Signal Length_Signal; 1/dt 1/dt 1/dt; Output];

cd ..

cd('SORTED RECORDS')

% writematrix(Acc,[filename(1:end-4),'.dat'],'Delimiter','tab');

writematrix(Acc,[filename(1:end-4),'.txt'],'Delimiter','tab');

cd ..

cd('MOTION RECORDS RENADIC_V2')

end

close(wb)

cd ..

%% function name: import_from_chile_v2

function [dt, time, Output, indexing] = import_from_chile_v2_corrected(filename)

stopper_line = '/& ----------';

len_stop_line = length(stopper_line);

channel_line = 'CHAN ';

len_chan_line = length(channel_line);

fid = fopen(filename,'r');

tline = fgetl(fid);

i=1;

tmp_1 = [];

tmp_2 = [];

tmp_3 = [];

component = 1;

component_internal = 0;

line_tmp = 0;

indexing = zeros(1,3);

support_vector = zeros(1,8);

dt_indicator = 0;

while ischar(tline)

i=i+1;

tline = fgetl(fid);

len_tline = length(tline);

if dt_indicator == 0

l1 = length('AT EQUALLY-SPACED INTERVALS OF ');

i1 = strfind(tline,'AT EQUALLY-SPACED INTERVALS OF ');

i2 = strfind(tline,' SEC.');

if isempty(i1) & isempty(i2)

else

dt = str2double(tline(i1+l1:i2-1));

dt_indicator = 1;

end

end

if len_tline>len_chan_line

if strcmp(tline(1:len_chan_line),channel_line)

if strcmp(tline(11),'E') || strcmp(tline(10),'T')

indexing(component) = 1;

elseif strcmp(tline(11),'N') || strcmp(tline(10),'L')

indexing(component) = 2;

elseif strcmp(tline(11),'V')

indexing(component) = 3;

end

end

end

%%%%%%% RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR

if i>46 && component == 1 && component_internal <= 2

% disp(['component = ',num2str(component)])

% disp(['component_internal = ',num2str(component_internal)])

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

component_internal = 0;

elseif length((strfind(tline,'OF VELOC D')))>0

component_internal = component_internal + 1;

elseif length((strfind(tline,'OF DISPL D')))>0

component_internal = component_internal + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if component_internal == 0

temp = str2num(tline);

support_vector(1,1:length(temp)) = temp;

tmp_1 = [tmp_1; support_vector];

support_vector = zeros(1,8);

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

elseif i>line_tmp+46 && component == 2 && component_internal <= 2

% disp(['component = ',num2str(component)])

% disp(['component_internal = ',num2str(component_internal)])

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

component_internal = 0;

elseif length((strfind(tline,'OF VELOC D')))>0

component_internal = component_internal + 1;

elseif length((strfind(tline,'OF DISPL D')))>0

component_internal = component_internal + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if component_internal == 0

temp = str2num(tline);

support_vector(1,1:length(temp)) = temp;

tmp_2 = [tmp_2; support_vector];

support_vector = zeros(1,8);

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

elseif i>line_tmp+46 && component == 3 && component_internal <= 2

% disp(['component = ',num2str(component)])

% disp(['component_internal = ',num2str(component_internal)])

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if len_tline > len_stop_line

if strcmp(tline(1:len_stop_line),stopper_line)

line_tmp = i;

component = component + 1;

component_internal = 0;

elseif length((strfind(tline,'OF VELOC D')))>0

component_internal = component_internal + 1;

elseif length((strfind(tline,'OF DISPL D')))>0

component_internal = component_internal + 1;

end

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if component_internal == 0

temp = str2num(tline);

support_vector(1,1:length(temp)) = temp;

tmp_3 = [tmp_3; support_vector];

support_vector = zeros(1,8);

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

end

end

%%%%%%% RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR

tmp_1 = tmp_1';

tmp_2 = tmp_2';

tmp_3 = tmp_3';

accel_factor = 1; % conversion from cm/s² to cm/s²

tmp = zeros(max([prod(size(tmp_1)),prod(size(tmp_2)),prod(size(tmp_3))]),3);

tmp(1:prod(size(tmp_1)),1)=tmp_1(:);

tmp(1:prod(size(tmp_2)),2)=tmp_2(:);

tmp(1:prod(size(tmp_3)),3)=tmp_3(:);

Output =[tmp(:,indexing==1) tmp(:,indexing==2) tmp(:,indexing==3)];

time = 0:dt:(length(tmp(:,1))-1)*dt;

end

[Mathieu NOE]

hello again

simply correct these lines (commented are original code)

    if len_tline>len_chan_line
        if strcmp(tline(1:len_chan_line),channel_line)
%             if strcmp(tline(11),'E') || strcmp(tline(10),'T') % no 
            if strcmp(tline(11),'E') || strcmp(tline(11),'T') % yes
                indexing(component) = 1;
%             elseif strcmp(tline(11),'N') || strcmp(tline(10),'L') % no
            elseif strcmp(tline(11),'N') || strcmp(tline(11),'L') % yes
                indexing(component) = 2;
            elseif strcmp(tline(11),'V')
                indexing(component) = 3;
            end
        end
    end

[Mathieu NOE]

a better and more robust approach, if the position of the letters (L,T,V,...) are not sure (or fluctuating

is to extract the portion of the line that contains these letters : we know that the letters are somewher between the : and the ( characters so we can take advantage of that information

newStr = extractBetween(tline,":","("); % get the portion of the line that contains the searched letters L,T,V, etc...

then I would suggest to use the function contains which make the code more compact and neater

the entire reworked portion of code is then :

    if len_tline>len_chan_line
        if strcmp(tline(1:len_chan_line),channel_line)
            newStr = extractBetween(tline,":","("); % get the portion of the line that contains the searched letters L,T,V, etc...
%             if strcmp(tline(11),'E') || strcmp(tline(10),'T')     % no 
%             if strcmp(tline(11),'E') || strcmp(tline(11),'T')     % yes
            if contains(newStr,'E') || contains(newStr,'T')         % more robust
                 indexing(component) = 1;
%             elseif strcmp(tline(11),'N') || strcmp(tline(10),'L')     % no
%             elseif strcmp(tline(11),'N') || strcmp(tline(11),'L')     % yes
            elseif contains(newStr,'N') || contains(newStr,'L')         % more robust
                indexing(component) = 2;
%             elseif strcmp(tline(11),'V')
            elseif contains(newStr,'V')         % more robust
                indexing(component) = 3;
            end
        end
    end

the same comment applies to another portion of code where you can improve :

here's one section I modified but you can apply the modifications to all similar portions of the code that follow

    if len_tline > len_stop_line
    if strcmp(tline(1:len_stop_line),stopper_line)
        line_tmp = i;
        component = component + 1;
        component_internal = 0;
%        elseif length((strfind(tline,'OF VELOC D')))>0   % ok but not optimal
       elseif contains(tline,'OF VELOC D') % simpler, neater
       component_internal = component_internal + 1;
%      elseif length((strfind(tline,'OF DISPL D')))>0   % ok but not optimal
       elseif contains(tline,'OF DISPL D') % simpler, neater
        component_internal = component_internal + 1;
    end
end

the entire file is in attachment

again you have to remove some of my comments in the first lines to get that portion of your code working again (simply for me I don't use them).

all the best

Answer 2

[Mathieu NOE]

hello

I created first some dummy files according to the list (would have been a bit easier with a txt file rather than a picture...)

just fyi this is a code to create some numerical arrays (as you didn't provide the data files) I will then process afterwards

a = readlines('list.txt')

for ck = 1:numel(a)
    filename = join([a(ck) '.txt'],'')
    data = rand(10+5*ck,3);
    writematrix(data,filename);
end

now this is the code for processing the data in the new /out folder (to avoid mixing input and output txt files)

path = pwd;  % current (your) folder 

S = dir(fullfile(path,'2021*.txt'));

%% export 

out_hne = [];
out_hnn = [];
out_hnz = [];
out_hne_filename = []; %  
out_hnn_filename = []; %  
out_hnz_filename = []; %  

for k = 1:numel(S)
    current_folder = S(k).folder;
    current_filename  = S(k).name;
    F = fullfile(current_folder,current_filename);
     
    % case 1 :  "HNE"
    if contains(current_filename,'HNE')
        data_hne = readmatrix(F);
        out_hne = [out_hne;data_hne]; % data vertical concatenation
        out_hne_filename = [out_hne_filename;current_filename]; % debug 
        
     % case 2 :  "HNN"
   elseif contains(current_filename,'HNN')
         data_hnn = readmatrix(F);
        out_hnn = [out_hnn;data_hnn]; % data vertical concatenation   
        out_hnn_filename = [out_hnn_filename;current_filename]; % debug 

      % case 3 :  "HNZ"
    elseif contains(current_filename,'HNZ')
         data_hnz = readmatrix(F);
        out_hnz = [out_hnz;data_hnz]; % data vertical concatenation   
        out_hnz_filename = [out_hnz_filename;current_filename]; % debug 
       
    end

end

%% exports to /out folder

out_folder = [current_folder '/out'];

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));
disp(' HNE files processed according to this list :');
disp(out_hne_filename)
disp(' -------------------------------------------');

writematrix(out_hnn,fullfile(out_folder,'all_HNN.txt'));
disp(' HNN files processed according to this list :');
disp(out_hnn_filename)
disp(' -------------------------------------------');

writematrix(out_hnz,fullfile(out_folder,'all_HNZ.txt'));
disp(' HNZ files processed according to this list :');
disp(out_hnz_filename)
disp(' -------------------------------------------');

in the command window you will see the order of processing the files :

 HNE files processed according to this list :
20211104-041733-C100-HNE.txt
20211104-041733-C200-HNE.txt
20211104-041740-C330-HNE.txt
20211104-041751-GO04-HNE.txt
20211104-041752-C160-HNE.txt
 -------------------------------------------
 HNN files processed according to this list :
20211104-041733-C100-HNN.txt
20211104-041733-C200-HNN.txt
20211104-041740-C330-HNN.txt
20211104-041751-G004-HNN.txt
20211104-041752-C160-HNN.txt
 -------------------------------------------
 HNZ files processed according to this list :
20211104-041733-C100-HNZ.txt
20211104-041733-C200-HNZ.txt
20211104-041740-C330-HNZ.txt
20211104-041751-GO04-HNZ.txt
20211104-041752-C160-HNZ.txt
 -------------------------------------------

[Jorge Luis Paredes Estacio]

I am using Matlab R2020a. This was the whole error message:

Error using writematrix (line 156)

Unable to open file 'E:\Civil Egineering PhD\Processing data matlab\matlab code\MOTION RECORDS CSN CHILE MEJORADO\out\all_HNE.txt' for writing:

No such file or directory

Error in Read_data_chile_joining (line 39)

writematrix(out_hne,fullfile(out_folder,'all_HNE.txt'));

>>

Answer 3

[Jorge Luis Paredes Estacio]

Thank you very much. I managed to upload some of the files.

[Mathieu NOE]

just fyi, please put comments in "comments" section , not in "answers"