Run a script on all files in a folder

I have data files in a folder that need editing (deleting columns, deleting some rows, renaming variables, renaming cells, adding variables based on filename, etc..).

I have already managed to piece together different scripts to get the above to be performed on one single file (here: MS01walnut.txt). I need the same things to be performed on all files in a folder. I have two questions I cannot seem to solve:

1. How can I apply the code automatically to all files in the folder? I assume this is "looping". I tried this piece of code, I'm not sure if it's all that is needed (I couldn't get Question 2 to be solved so I couldn't really test it):

files = dir('*.txt');
for file = files'
    txt = load(file.name);

2. What changes do I need to make to my code to make it responsive to the loop?

Below is my code:

inputdir = 'I:\Side projects\Holly study_follow up_MSc Supervision\Data analysis_Matlab\Raw'
outputdir = 'I:\Side projects\Holly study_follow up_MSc Supervision\Data analysis_Matlab\Output'

clear all
%Loop through the folder (i'm not sure this is working.. shouldn't there be
%anything else added?)
files = dir('*.txt');
for file = files'
    txt = load(file.name);
    % Do the following stuff
%Import data 
MS = readtable('MS_01_walnut.txt','Delimiter','\t','ReadVariableNames',false)
%delete unnecessary tables and columns
        MS([1,2,78],:) = [];
        MS(:,[1,2,5,6,8,9,10,11,12,13,14,15,16,17,18,19,20,21,23]) = [];

%change header names
        MS.Properties.VariableNames{'blocknumber'} = 'phase'
        MS.Properties.VariableNames{'trialnumber'} = 'trialphase'
        MS.Properties.VariableNames{'trialcontents'} = 'stimulus'
        MS.Properties.VariableNames{'VarName22'} = 'rating'
%turn filename into a variable and extract subject number from it

filename = 'MS_01_walnut'
ppnr = regexp(filename,'[0-9]{2}','match','once')

%turn ppnr into vector of 75 lines and stick it to the table

ppnr75 = repmat(ppnr,75,1)
MS.subj = ppnr75

%find the CSM
whatisCSM = any(table2array(MS(1:6,3)) == 35)

%perform series of replacements
        %if whatisCSM = 1, then replace: 40(gen1), 45(gen2), 50(gen3),
        %60(gen4), 65(gen5), 70(gen6), 75(gen7)
        %if whatisCSM = 0, then replace: 35(gen7), 40(gen16), 45(gen5), 50(gen4),
        %60(gen3), 65(gen2), 70(gen1)
    if whatisCSM == 0     
    zlookup = {'gen7', 'gen6', 'gen5', 'gen4', 'gen3', 'gen2', 'gen1', 'CSM', 'CSP'};
    MSz = MS{:,3};
    [tf, idx] = ismember(MSz, [35, 40, 45, 50, 60, 65, 70, 75, 55]);
    out = cell(length(tf),1);
    out(tf) = zlookup(idx(tf));
    out(~tf) = sprintfc('%g', MSz(~tf));
    MS{:,3} = out

      else 
      zlookup = {'gen7', 'gen6', 'gen5', 'gen4', 'gen3', 'gen2', 'gen1', 'CSM', 'CSP'};
      MSz = MS{:,3};
      [tf, idx] = ismember(MSz, [75, 70, 65, 60, 50, 45, 40, 35, 55]);
      out = cell(length(tf),1);
      out(tf) = zlookup(idx(tf));
      out(~tf) = sprintfc('%g', MSz(~tf));
      MS{:,3} = out
      end 

%writetable into the 'Output' directory 
writetable(MS,'MS_01_walnut_mat.txt','Delimiter',' ')
end

Right now the code works as needed on one file but I really need to understand how to automatically apply it to all files in a folder. Thanks in advance for any tips!

Answer 1

[Stephen23]

You need to include the required directory path whenever you do anything with files that are not in the current directory, it is recommend to use fullfile to create the required filename. While looping over the elements of the structure will work, it is standard practice to use an index.

...
S = dir(fullfile(inputdir,'*.txt'));
for k = 1:numel(S)
    fnm = fullfile(inputdir,S(k).name);
    txt = load(fnm);
    ...
end

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

[Boushra Dalile]

thanks! could you please clarify where this would go in the code above? should any additional changed be made to the code when implementing the index?

[Stephen23]

"could you please clarify where this would go in the code above?"

Your code has one for loop, my code has one for loop. It is the same for loop. The ellipses indicate the rest of your code.

"should any additional changed be made to the code when implementing the index?"

Just what I showed you.