Recognizing and removing unwanted lines from data file

Hey all,

After way to many hours of trial and error, and looking on the web for an answer I've ran out of options. I'm pretty sure you guys should be able to solve this problem for me since it doesn't seem that complicated.

I have a couple thousands of data files with a format like this:

I want Matlab to give me the values in the matrices, (so in the first matrix [0.75 55.0; 1.00 55.0; 1.25 86.3].

I've succeeded in writing code that will give me the first matrix, but i havent been able to textscan in such a way that it skips the unwanted lines.

d=[];ssc=[];
files = dir([datdir 'MV2*']);   
for j=1:length(files)
  fid = fopen([datdir files(j).name],'rt');
  C=textscan(fid,'%f %f','Delimiter','\n');
  fclose(fid);
  d=[d;C{1}];
  ssc=[ssc;C{2}];
end

Some advice or a solution would be amazing! Thanks in advance :)

Answer 1

[dpb]

for j=1:length(files)
  fid = fopen([datdir files(j).name],'rt');
  d=[d;cell2mat(textscan(fid,'%f %f','headerlines', 2, ...
                                     'commentstyle', '*', ...
                                     'collectoutput',1));
  fclose(fid);
end

Untested; much better posting technique would be to attach a short section of the file itself rather than an image so folks can test directly...

[Yuri Engelshoven]

Thanks a lot for your response. I'll keep that in mind for the next time. I tried copying some of the file in here but the layout wouldn't work through in the actual post.

I've already solved the issue by the use of some basic functions:

for j=1:length(files)
  fid = fopen([datdir files(j).name],'rt');
  for p=1:10
    tmp=fgetl(fid);
  end
  for i=1:100
    for j=1:6
      tmp=fgetl(fid);
    end
    C=textscan(fid,'%f %f');
    d=[d;C{1}];
    ssc=[ssc;C{2}];
  end
  fclose(fid);

What does cell2mat do in your script?

[dpb]

cell2mat simply gets rid of the cell array by converting it to an array; same as "the curlies" in your subsequent step. It just eliminates the temporary cell array.

To add data either

• use the paper clip to attach the file itself or
• if paste a section format it as code to retain formatting

It's a real annoyance that TMW can't seem to fix it to get rid of the automatic word wrapping--for a computing forum it's just the wrong choice for a data entry default.