FGETL taking a long time to execute

I am running fgetl in a loop similar to the code below. When profiling the code, I noticed that fgetl(fid) takes the most time to execute compared to the rest of the code. After checking the fgetl documentation, I found the following note in the documentation:

fgetl is intended for use with files that contain newline characters. Given a file with no newline characters, fgetl may take a long time to execute.

However, the documentation does not provide a solution for files without newline characters. Since my file lacks newline characters, what would be the best alternative to fgetl?

fid = fopen(FileName);

while ischar(tline) %loops over 1e5 times
    % read new line
    tic
    tline = fgetl(fid); 
    toc
    % Do stuff with tline
    
    
end

Answer 1

[Matt J]

readlines?

[ROYGBIV]

readlines reads the whole file doesnt it? I still want to keep te abilty to read line by line just like fgetl

[Steven Lord]

I still want to keep te abilty to read line by line

How specifically do you define what a line is if your file has no newline characters? Are you sure that one fgetl call isn't reading in the whole file, treating it as one (potentially very long) line?

[Matt J]

I still want to keep te abilty to read line by line just like fgetl

If you are only reading a few lines, I wonder why speed matters. If you are reading in a substantial percentage of the file, I wonder if it wouldn't be faster just to read the whole thing in in a single function call and discard what you don't use.

[ROYGBIV]

Unfortunately I cannot share the file, but here is the summary of the code profile. Each line in the file is slightly unique therefore I need to interpret/parse each line one at a time. I could read in the whole file but that would use up too much memory.

fgetl does return a the desired lines but does so at a much slower excution time compared to the other functions I am calling within the loop.

Could it be reading from a file takes long and I have to live with it?

[Walter Roberson]

You could experiment with using

cell2mat(textscan(FID, '%[^\n]', BUFFERSIZE))

to read BUFFERSIZE lines. It might potentially be faster than calling fgetl() BUFFERSIZE times, due to reduced overhead.