Missing rows using Textscan delimiter

Hi! I have a tiny problem when importing .txt-files using textscan.

The data contains strings ending with '=' and I'm trying to use this syntax to extract my data:

disp('Collecting TAF information, this may take a while... :)')
TI = duration(timein,'InputFormat','hh:mm');
TO = duration(timeout,'InputFormat','hh:mm');
P = '\\winfs-lang\data\lang\prod\flygarkivet\TAF';
S = dir(fullfile(P,'*.txt'));
D = regexp({S.name},'\d{10}','match','once');
T = datetime(D, 'InputFormat','yyMMddHHmm');

X = isbetween(T,DS,DE);
tod = timeofday(T);
Y = tod>=TI & tod<=TO; % time of day
Z = S(X&Y);

numfilesT = length(Z(:,1)); %create empty cell
DataT = cell(1, numfilesT); %create empty cell
%Imports all .txt files according user input time and convert into strings
for hh = 1:numfilesT
    fnm = fullfile(P,Z(hh).name);
    fileIDT = fopen(fnm);
    DataT{hh} = textscan(fileIDT,'%s','delimiter','=','headerlines',1); %read all characters in fileID 
    fclose(fileIDT); %close fileID
end

The data looks like this (also attached):

FCSN33 ESPA 070200 AAA

TAF AMD ESPA 070500Z 0705/0712 02005KT 9999 -SN SCT002 BKN015

PROB40 0705/0712 3000 SN VV006 RMK MIL=

and

FCSN33 ESPA 070200 CCA

TAF COR ESPA 080400Z 0804/0812 36005KT 9999 -SN BKN025

PROB40 0804/0812 2500 BKN007 RMK MIL=

The end result is looks like this:

TAF AMD ESPA 070500Z 0705/0712 02005KT 9999 -SN SCT002 BKN015

TAF COR ESPA 080400Z 0804/0812 36005KT 9999 -SN BKN025

It seems that I'm missing the second line of data in some of the files. This problem does not occur in every case.

Can you please help me with sorting this problem?

Best regards

Linus

Answer 1

[Walter Roberson]

Reminder: textscan %s format skips all leading whitespace, including tabs and linefeeds . You have to be quite careful if you are trying to use %s to read multiple lines while preserving spaces -- you would have to tell textscan to change its Delimiter and its Whitespace.

Basically, if you need to read whitespace intact then use a %[^\n] -- or in your case perhaps %[^=]

Frankly, I would not bother using textscan() in this case. Instead I would suggest using fileread() to get the file as a character vector, and then use tools such as regexp() to pull out the relevant parts.

[Linus Dock]

Thank you, I will try another way of doing this.

/Linus