h5read variables within table (matlab class table) from .mat files not working

Reading (numerical, logical, char) variables from nested structs in .mat files with h5read works fine. It saves a lot of time if you only need a few variables from a big (> 1 GB) .mat file.(Loading a 20 GB .mat file takes very, very long. If it's compressed even longer.) (I store big data within one big nested structure, that is saved in a .mat file. It contains all kinds of variables and datatypes. Among them nice tables.)

I can't get it to work to h5read variables within table (matlab class table) from .mat files.

2 questions:

1. Has anyone got a solution for this problem using h5read? (I already know I could workaround via table2struct(tbl) before saving)
2. Are there alternatives to quickly read variables from a table (or the whole table), that is deeply nested within a struct in a .mat file?

[Siddharth Bhutiya]

Can you post a code sample and example data that people can try. When you say I can't get it to work for tables, what did you try and what is the error that you are seeing?

[Janis]

a.nested.structure.dbl = (1:10)';

a.nested.table = struct2table(a.nested.structure);

a.nested.table.dbl

fileName = 'vars.mat';

save(fileName)

% works fine

dbl = h5read(fileName, '/a/nested/structure/dbl')

% errors

dbl = h5read(fileName, '/a/nested/table/dbl')

Error using h5readc

The HDF5 library encountered an error and produced the following stack trace information:

H5O_msg_read_oh message type not found

H5O_msg_read unable to read object header message

H5G__stab_lookup can't read message

H5G__obj_lookup can't locate object

H5G_traverse_real can't look up component

H5G_traverse internal path traversal failed

H5G_loc_find can't find object

H5Dopen2 not found

Error in h5read (line 66)

[data,var_class] = h5readc(Filename,Dataset,start,count,stride);

Thanks for your repsonse!

[Janis]

The printed error was from 2019b .. but I think the problem is the same in 2021b :

Error using h5readc

Unable to open the file because of HDF5 Library error. Reason:Unknown

Error in h5read (line 93)

[data,var_class] = h5readc(Filename,Dataset,start,count,stride);

Answer 1

[Shubham]

Hi Janis,

Regarding your first question, it is possible to use the h5read function to read variables within a table that is deeply nested within a struct in a .mat file. However, it requires some knowledge of the file's structure and careful use of the h5info function to determine the path to the desired variable. Here is an example code snippet that demonstrates how to read a variable named "myVar" from a table that is nested within a struct in a .mat file:

% Open the .mat file using the h5info function
matInfo = h5info('myFile.mat');

% Determine the path to the desired variable
tablePath = '/myStruct/myTable';
varPath = [tablePath '/myVar'];

% Use h5read to read the variable
myVar = h5read('myFile.mat', varPath);

Regarding your second question, there are several alternatives to quickly read variables from a table (or the whole table) that is deeply nested within a struct in a .mat file. Here are a few options:

1. Use the load function with the '-mat' option to load the entire .mat file into memory. This can be faster than using h5read for small files, but for larger files it may still be too slow.
2. Convert the table to a struct using the table2struct function before saving it to the .mat file. This will allow you to use h5read or other functions to read the variables more easily.
3. Split the data into smaller .mat files based on your analysis needs, rather than saving everything in one large file. This can improve read and write times.
4. Use a different file format that is better suited for large data, such as HDF5 or Parquet.

Ultimately, the best solution will depend on the size and complexity of your data, as well as your specific analysis needs.

[Janis]

Hi Shubham,

thank you for your response.

The exemplary code you posted doesn't work for me.

Do you really have a working example?

I experimented with this, but i couldn't get the original data:

a = struct;

strct = struct('dbl', (1:10)');
% a.nested.structure.dbl = (1:10)';
a.nested.tbl = struct2table(strct);
a.nested.tbl.dbl
fileName = 'vars.mat';
save(fileName, 'a')


matInfo = h5info(fileName)
 
 
 h5disp(fileName)

% i can read something, but it is only 6 entries long and doesn't seem to have anything to to do with the original data ?!
tblItself = h5read(fileName, '/a/nested/tbl')

 % this errors
dblFromStruct = h5read(fileName, '/a/nested/tbl/dbl')

[Janis]

but what's interesting is that h5disp knows there is a table:

I

    Group '/a' 
        Attributes:
            'MATLAB_class':  'struct'
        Group '/a/nested' 
            Attributes:
                'MATLAB_class':  'struct'
                'H5PATH':  '/a'
            Dataset 'tbl' 
                Size:  6x1
                MaxSize:  6x1
                Datatype:   H5T_STD_U32LE (uint32)
                ChunkSize:  []
                Filters:  none
                FillValue:  0
                Attributes:
                    'MATLAB_class':  'table'
                    'MATLAB_object_decode':  3
                    'H5PATH':  '/anested'

[Janis]

I know that h5read is not trivial. It finds a bytestream. E.g. yo have to typecast chars, that are 8bit intergers with `char()`.

i just wonder if there is a way to read a variable within a table. Just like you would do with a struct. Which works totally fine.