fileDatastore error in @matlab.io​.datastore​.internal.​pathLookup​Local: The logical indices contain a true value outside of the array bounds

Using R2020b on a linux (Red Hat 8) system, I am trying to set up a datastore for several, locally stored data repositories using

filedatastore(data_path,...
    'ReadFcn',@custReadFcn,'I'),...
    'IncludeSublfolders',true,...
    'UniformRead',false,...
    'FileExtensions','.ext');

Some of the datastores work just fine. But others give me an error "The logical indices contain a true value outside of the array bounds." I traced the error to a function in

FileBasedDatastore.m

line 305.

[loc, fileSizes] = pathLookup(loc, nvStruct.IncludeSubfolders, noSuggestionInEmptyFolderErr);

Inside this function on line 133, which reads

[f{ii}, s{ii}] = localLookupFcn(pth, includeSubfolders);

I occaisonally get numel(f) ~= numel(s). So when f and s are returned as loc and fileSizes from pathLookup, and I get to the switch statement at line 327, I get the error mentioned above when MATLAB tries to run

fileSizes = fileSizes(filterExts);

Because the sizes of fileSizes and filterExts don't match. There is no documentation for localLookupFcn, and when I try to open the function, MATLAB claims it doesn't exist.

I know that some of the files were manually deleted in the repositories where the errors occur. I'm guessing somehow, MATLAB still sees some of them or some symbolic links. I don't understand why I am running into an error, why MATLAB hangs up on the size of files, how it's finding a different number of files and file extensions, or what to do to get around this. Any ideas?

Answer 1

[Luke Hubbard]

It turns out there were broken symbolic links to data-generating files in some of the subdirectories from an earlier renaming of parent directories, which was necessary to do so that labels could be applied automatically. In any case, removing the broken symbolic links fixed the issue. In linux this can be done using

find /path/to/directory -maxdepth 1 -xtype l -delete

as described at this site https://linuxize.com/post/how-to-remove-symbolic-links-in-linux/