MATLAB Answers

Why is readcell not functioning as expected?

5 views (last 30 days)
Antoon De Cleen
Antoon De Cleen on 19 Nov 2020
Commented: Antoon De Cleen on 23 Nov 2020
We have an application that watches a directory and looks for .csv files being created there. Upon detecting a new file, the csv is read using readcell and a function is called using the arguments found in the csv.
Almost always this works just fine, but on some rare occasions it seems like readcell does not return anything.
The code it executes in is really simple:
% Read file
try
if LOGGING
logging_message = sprintf('Trying to read command csv');
write_info(diary_file_id, logging_message);
end
% Try to get directory name and object type from input trigger file
input_csv = readcell(txt_filename);
dir_to_process = input_csv{5};
type_to_process = input_csv{6};
% Replace escape characters
dir_to_process = replace(dir_to_process, '\', '/');
if LOGGING
logging_message = sprintf('Command csv gave dir_to_process: %s and type_to_process: %s', dir_to_process, type_to_process);
write_info(diary_file_id, logging_message);
end
catch ME
% Write error message to diaryfile
logging_message = sprintf('Failed with error message: %s',ME.message);
write_error(diary_file_id, logging_message);
write_error(diary_file_id, getReport(ME, 'extended', 'hyperlinks', 'off'))
if DEBUG
rethrow(ME)
end
end
As you can see it writes output to a logfile. Such a logfile looks like this when the problem occurs:
2020-11-16_17.58.21.879 MATLAB INFO | Eventhandler received a file
2020-11-16_17.58.21.894 MATLAB INFO | Received command file name 2020-11-16_17.49.51.csv
2020-11-16_17.58.26.917 MATLAB INFO | Trying to read command csv
The program does not return and does not return an error because those are also logged.
This is what an error can look like in the logging file for instance with an empty csv file:
2020-11-18_09.00.32.929 MATLAB INFO | Eventhandler received a file
2020-11-18_09.00.32.934 MATLAB INFO | Received command file name empty.csv
2020-11-18_09.00.37.940 MATLAB INFO | Trying to read command csv
2020-11-18_09.00.37.979 MATLAB ERROR | Failed with error message: Index exceeds the number of array elements (0).
2020-11-18_09.00.37.985 MATLAB ERROR | Index exceeds the number of array elements (0).
Error in eventhandler_changed (line 37)
dir_to_process = input_csv{5};
In order to recreate this problem I've tried to:
  • Open the file in a different program simultaniously
  • Pass bogus arguments in the csv
  • Pass no arguments in the csv
But with no succes. I've also tried looking at the readcell source code but this is all I can see:
try
func = matlab.io.internal.functions.FunctionStore.getFunctionByName('readcell');
C = func.validateAndExecute(filename,varargin{:});
catch ME
throw(ME);
end
Which clearly shows that an error should be throwns if it crashes, but it definitely does not seem to do that in my case.

  1 Comment

Antoon De Cleen
Antoon De Cleen on 19 Nov 2020
For those interested, the version this is compiled on: MATLAB Version: 9.7.0.1247435 (R2019b) Update 2
So far I've only seen this problem happen while running the executable and not while running in Matlab.

Sign in to comment.

Answers (1)

dpb
dpb on 20 Nov 2020
Edited: dpb on 20 Nov 2020
If the creating and reading routines don't have some sort of semaphore arrangment to let the one know the file is closed it's possible as you've seen that the OS has created the file but not yet flushed output buffers to have actually written the file content to disk.
This is the symptom you're seeing; it is much less likely to occur in a MATLAB session owing to the extra overhead of the interpreter, etc.
The ideal solution is a semaphore if you have both pieces of code under your control/development; otherwise, probably the best you can do is to test for the empty result in the reader and spin inside there for a delay to let the OS catch up.

  3 Comments

Antoon De Cleen
Antoon De Cleen on 20 Nov 2020
Thanks for the input. We did indeed run into that problem a while back. We did get an error along the lines of unable to access file back then, which is not something that we see in this case.
However, we now create the file in a temporary folder until it is written then move it over to this folder. We also pause for 5 seconds after receiving before we start reading. Do you think that with these measures that this is still the issue?
Creation of the file is not under matlab control so any kind of communication between processes would also defeat the purpose of using this file based part. If this remains an issue, however, this is a viable option.
As for testing for an empty result; I've now created a very simple implementation of a csv parser that just reads the text file line per line and parses comma separated values to a struct. It's basic but it does work and give us more control over the reading process. Time will tell wether or not this solves the issue.
dpb
dpb on 20 Nov 2020
Well, it's not identically the same, but the same idea. You can still on occasion find the file directory entry before the OS completes the operation.
If that does happen then the handle returned will remain attached to that entry even while you wait.
I think the way would be to, when the new file entry is found, CLOSE that handle and wait, THEN reopen the file.
I believe that would work quite reliably, and probably with not a very long wait period. But, to release the handle first would be the key.
While working this over, try a diagnostic--return the dir() information and check the .bytes entry to see if, in fact, the OS will return 0. I'm guessing it will and you can write logic to test that before opening and bypass the alternative reading routine you presently have.
If that turns out to be so, you can simply sit in an idle loop checking until the indicated size is nonzero.
Antoon De Cleen
Antoon De Cleen on 23 Nov 2020
Thanks for the suggestion, I'll add a checking loop for the size check. If anything changes I'll let you know.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!