MATLAB Answers

readtable not understood behaviour

12 views (last 30 days)
Cristian Berceanu
Cristian Berceanu on 6 Sep 2019
Commented: dpb on 9 Sep 2019
Hi all,
I have the following settings.ini file (its a basic text file):
;delay of calibration start (in seconds) after scanning the DUT serial number
intCalibrationStartDelay = 5
;directory path at which the log files are saved
strSaveFilePath = ''
;COM port of the power meter
strPowerMeterCOMPort = COM4
I would like to read it as a table, with each line of it becoming an element in my table, and for this, I use the following:
SettingsData = readtable('settings.ini',"FileType","text","Delimiter",'\r\n',"ReadVariableNames",false,"ReadRowNames",false,"TextType",'string');
For some reason, however, the first two lines of the file do not get imported, any idea why? The result of the instruciton above is always:
SettingsData =
6×1 table
";directory path at which the log files are saved"
"strSaveFilePath→→→= ''"
";COM port of the power meter"
"strPowerMeterCOMPort→→= COM4"
Best regards,

Accepted Answer

dpb on 6 Sep 2019
You could make a textimportoptions object and use it with readtable but with the text file as text, I'd just scan it and then convert...
readtable is designed specifically for tabular datafiles and works really, really hard to import everything to fit the model of a header line followed by tabular (mostly numeric) data. You have to fight against that builtin prejudice to overcome the internal evaluations done to try to interpret what the file structure is. textscan just reads in what it sees.
My $0.02 for personal preference for the particular case.
dpb on 6 Sep 2019
"file here is simply not tabular data (in the meaning used by readtable)."
That's precisely the prejudice with which readtable was designed was speaking of... :)
Definition of prejudice
2a : preconceived judgment or opinion
readtable has a preconceived idea of what constitutes a table that comes from the design ideas implemented in the table class.
Granted, it has a general connotation of bad; I chose it here not out of hostility or the like but as simply editorial license...

Sign in to comment.

More Answers (2)

Jeremy Hughes
Jeremy Hughes on 6 Sep 2019
A simpler way to do this would be:
lines = splitlines(fileread(filename))
  1 Comment
Walter Roberson
Walter Roberson on 6 Sep 2019
Or if, like me, you have...older... habits,
lines = regexp( fileread(filename), '\r?\n', 'split');
Both splitlines and regexp 'split' used this way have an oddity of often returning an empty character vector or string element at the end. This occurs for files whose last line ends in newline, in which case regexp and splitlines both create an entry for the emptiness between the newline and the end of file. This is not wrong, but it can be unexpected.
For text files, none of the major operating systems strongly define whether newline is a line separator or a line terminator. If newline is a line terminator then the newline at the end of the last line would terminate that line and then there would be no following lines so you would not expect any empty string at that point. However, if newline is a line separator then the newline at the end of the last line is separating that line and the empty line up to end of file and it makes (some) sense for there to be an empty string at that point.
The philosophic difference shows up in the treatment of a last line that has no newline and just suddenly ends at end of file: is that a malformed line (if newline is the line terminator then it "should" have appeared there to terminate the line before end of file), or is it a valid line (if newline is a line separator then it is not needed to separate the end of the current line from end of file.)
Historically, Windows and Unix have both weakly defined newline as being a line separator -- that is, that it is considered entirely valid for a line to just suddenly end with end of file following. I say "weakly defined" because so many "standard" programs have gotten it wrong over the years, and even the fundamental C library routine fgets() implicitly requires that you test every time whether you actually received the trailing line terminators or not, which hardly anyone does...

Sign in to comment.

Cristian Berceanu
Cristian Berceanu on 9 Sep 2019
Thank you all for your suggestions. I piked the one that suits my needs best. Although, Imust admit I am still a bit puyyled as to why the readtable function does not operate as expected in this particular case.
Best regards,
dpb on 9 Sep 2019
"am still a bit puyyled as to why the readtable function does not operate as expected "
As Guillaume and I both noted, readtable is designed specifically for tabular data and does its darndest to force whatever it sees into what it thinks a table should look like.
When you give it something that isn't really tabular, "all bets are off!" as to what it may try to make of it.
Here the empty rows at the beginning were such that those first lines were interpreted as non-data lines.
Run detectImportOptions on the file and you can inspect how your particular file was interpreted by the internal heuristics and, if you really, really feel compelled to use readtable directly, you can edit the returned importoptions object to specify explicitly the starting row and then use the modified object with readtable to force the desired interpretation.
I think it's still simpler to just scan and then convert, but if you're doing this on a lot of files you can save the options object for reuse.

Sign in to comment.




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!