Convert data form .txt file into .mat file

Hey guys, hope you doing well.

I need help converting the data of hundreds of .txt files into one matlab file. Each text file has 3 columns. I was thinking about extracting the data of each column and then save it as a variable into one .mat file. In the end i would like to have one matlab file with 3 columns/variables.

This is the code that i have, but its not working for my needs. I'm having problems extracting the data of each column.

Files_September = dir('202109*.txt')
files = numel(Files_September);

for k = 1:files
    files2 = Files_September(k).name;
    fid = fopen(files2);
    data = fscanf(fid, '%c') 
    fclose(fid);
    save('September','data')
end

I'm attatching one text file so that you know what type of files im working with.

Thank you for your help.

Answer 1

[Voss]

Each time through the loop data contains the data from the current file, so that when you do save, only the current file's data will be saved in the mat-file, overwriting what was in the mat-file before (if anything). If you want one mat-file with the data from all the text files, you can accumulate the data in the loop and then save it once after the loop.

Files_September = dir('202109*.txt')
files = numel(Files_September);
data = '';
for k = 1:files
    files2 = Files_September(k).name;
    fid = fopen(files2);
    new_data = fscanf(fid, '%c');
    fclose(fid);
    data = [data new_data];
end
save('September','data')

Now data will be a character array of all the text in all the files and it will be stored in the file 'September.mat'. Maybe this is sufficient and maybe not; I don't know what kind of data is in those text files. Each file contains three columns, but three columns of what? Numbers? Names of countries? And are there line(s) of text at the top with column titles? The answers to those types of questions would determine which method is best for reading the files and accumulating the data properly.

Can you attach an example text file to your question so that we may help further? Thanks.

[Tiago André]

Thank you for your help.

Im attaching one text file. I need to import the data of multiple text files like this one. It's mostly numbers, being the first line the name of the variables only. I need the .mat file to be like this.

[Voss]

Thanks for the sample file. I'll modify the way the file is read, so that the program skips the one header line and stores the data as a matrix of numbers with three columns:

Files_September = dir('202109*.txt')
files = numel(Files_September);
data = [];
for k = 1:files
    files2 = Files_September(k).name;
    fid = fopen(files2);
    fgetl(fid); % skip the first line
    new_data = fscanf(fid, '%f'); % read data as floating-point numbers (not chars)
    fclose(fid);
    new_data = reshape(new_data,3,[]).'; % reshape to 3-column matrix
    data = [data; new_data];
end
save('September','data')

Now September.mat will have a variable called 'data' which is a three-column matrix of numbers containing the data from all the text files.

Note that the columns of data are not separated out into three separate variables, because I didn't know if that's a requirement or if it was just one way you were thinking about doing it. I think one variable with three columns makes more sense.

[Tiago André]

Thank you!

That's not a problem, it can be that way.

The code is working fine, i think. However, i'm getting a problem opening the data. it gave me a array with 118809600x3 double which is odd, i dont have that much data (at least it didnt show that much when i run the code i had). When i try to see the data, it says im out of memory.

[Tiago André]

I've found a solution for this. I was using the variable 'data' instead of the 'new_data' in the september file. Thank you for help!

I just have one question. Being the data saved as one variable, instead of 3, i can still use just one or different columns for graphics and data analysis, right?

[Voss]

Correct. You can use any or all columns for whatever you want to do.