Timetables inside table are not updating correctly

Hello,

I'm trying to fill a table with timetables and scalar values (for Diagnostic feature Designer APP input) and I'm having some troubles defining the table.

I 'm using the following to create the table; where MatStruct is a structure (output of dir()) that contains the directories and filenames of the .mat files that I want to load and read.

sz = [length(MatStruct),4]; 
varTypes = {'timetable','timetable','timetable','double'};
DataTable = table('size',sz,'VariableNames',{'channel1','channel2','channel3','ConditionVar'},'VariableTypes',varTypes);

At this point the table is created properly, so I start to load and read the actual data onto the table:

for i=1:length(MatStruct)
    
    load(fullfile(MatStruct(i).folder,MatStruct(i).name))
    dt = diff(TIME);
    dt = dt(1);
    ch1 = array2timetable(channel1,'SampleRate',1/dt);
    ch2 = array2timetable(channel2,'SampleRate',1/dt);
    ch3 = array2timetable(channel3,'SampleRate',1/dt);

    DataTable(i,1) = {ch1};
    DataTable(i,2) = {ch2};
    DataTable(i,3) = {ch3};
    DataTable(i,4) = {ConditionVar};
    
end

For some reason, the first 3 columns (timetable columns) of the table are not being updated with ch1, ch2 and ch3; they keep the inital 1x0 timetable that appeared once the table was created. The fourth column scalar values, 'ConditionVar', is working fine (code not shown).

ans =

  table

       channel1    
    _______________

    [1×0 timetable]

Whereas, if create the table as

    DataTable = table()

The table ends up with the correct values, but I can't name table's variables so it is a bit messy when using the APP.

ans =

  table

           Var1       
    __________________

    {5383×1 timetable}

I would like to understand why my created timetables are not being updated onto the table.

Thank you very much,

[dpb]

Why are you trying to put timetables inside tables, to begin with????

Are there disparate times in these timetables, and if so what is the intent in the usage in putting them into one table? If they are separate tests for somesuch, I'd think probably more logical would be to create an array of timetables instead rather than multiple timetables inside a table.

BUT, if you do this, the variable name in the table will be that of the timetable and will return the timetable object--you then have to reference into that timetable with whatever are the actual variables in the timetable.

Seems like I did something like that once accidentally not realizing it was even possible; I'll have to puzzle out the syntax for references again as I don't recall just how that went.

While it was cute in some ways to see a nested table structure, it wasn't anything really of in help at that time so I didn't play with it much; without an explanation for the intended use here I really don't see why to do it here, either.

[dpb]

But, I happened to still have in memory at timetable from a previous Answer, so--

> whos T*
  Name                Size             Bytes  Class        Attributes

  TT                289x1               5614  timetable              
  TT_15            8760x2             211438  timetable              
  TT_HHWL15          12x4               2124  timetable              
  TargetSheet         1x1                     handle                 
  Tw                  1x1                  8  double                 

>> T=table(TT);
>> head(T)
ans =
  8×1 table
                  TT              
            Time            Weight
    ______________________________
    01-Jan-2021 00:00:00    1348.8
    01-Jan-2021 00:15:00    1356.3
    01-Jan-2021 00:30:00    1354.6
    01-Jan-2021 00:45:00    1350.1
    01-Jan-2021 01:00:00    1341.4
    01-Jan-2021 01:15:00    1367.7
    01-Jan-2021 01:30:00      1382
    01-Jan-2021 01:45:00    1384.2
>> ans.TT.Weight
ans =
   1.0e+03 *
    1.3488
    1.3563
    1.3546
    1.3501
    1.3414
    1.3677
    1.3820
    1.3842
>> 

Yeah, that reminded me -- you dereference the variables similarly as to a nested stuct--just keep stringing dot names together to get where need to go.

Remember that you've encapsulated the timetable inside the variable in the table...whether that's a good idea or not would need to know, as noted above, just what the intent is and what the content of these various timetables is.

I would still think it's likely not the most appropriate storage layout, but could be convinced if can make a case for the why, other than just not doing what you really intended to do like my earlier example.

[pcm314]

Hello @dpb,

The reason of putting timetables (followed by a condition varible which can be a scalar or a categorical variable) inside a table is because, as far as I know, is te best way to input data to the Diagnostic Feature Designer APP of the Predictive Maintenace toolbox.

Here you can see how input data should look like:

Import and Visualize Ensemble Data in Diagnostic Feature Designer - MATLAB & Simulink - MathWorks España

Thank you very much

[dpb]

With R2020b here I can't find any syntax that will let me enter a timetable of multiple rows into a table as a collapsed variable with only one auxiliary variable the way that example shows.

Which release do you have/does the sample app require? I didn't go searching, not having that app, but is there another page that goes with it that shows how that sample dataset was created?

Whatever I do, I run into the problem

>> T=table('Size',[1,2],'VariableTypes',{'timetable','double'})
T =
  1×2 table
         Var1          Var2
    _______________    ____
    [1×0 timetable]     0  
>> T(1,1)=tmp
To assign to or create a variable in a table, the number of rows must match the height of the table. 
>> whos tmp
  Name      Size            Bytes  Class        Attributes

  tmp       8x1              1118  timetable              

>> T=table(tmp,0,'Size',[1,2],'VariableTypes',{'timetable','double'})
Error using table (line 233)
All table variables must have the same number of rows. 
>> 

It will create a composite datatable type, but can't store a real timetable into it; if embed the timetable as above it is contained in the table as a variable but still of the same height.

I don't see how to build the table they show, sorry...

Answer 1

[Cris LaPierre]

I think the issue is that you are treating tables like cells. In a cell, each can be a variable. However, in a table, an entire column represents a variable. You cannot, therefore, assign a table to a single element (row) of a column. It must be assigned to the entire column.

dt = 0.05;
channel1 = rand(3,2);
channel2 = rand(3,2);
channel3 = rand(3,2);

ch1 = array2timetable(channel1,'SampleRate',1/dt);
ch2 = array2timetable(channel2,'SampleRate',1/dt);
ch3 = array2timetable(channel3,'SampleRate',1/dt);

% Notice the variable names, ch1-3, across the top of the table
% The timetable variable names are displayed under the table variable names
% The timetable values are shown in their corresponding columns under ch1,
% ch2, and ch3
TT = table(ch1,ch2,ch3)

TT = 3×3 table
                   ch1                                   ch2                                   ch3                
      Time      channel11    channel12      Time      channel21    channel22      Time      channel31    channel32
    __________________________________    __________________________________    __________________________________

    0 sec        0.26206      0.96846     0 sec       0.049802       0.6536     0 sec         0.1514      0.30417 
    0.05 sec     0.29422      0.23072     0.05 sec     0.70208      0.07587     0.05 sec     0.81389      0.17342 
    0.1 sec       0.5714      0.26852     0.1 sec     0.012769      0.16812     0.1 sec     0.011715      0.36192 

I think you need to rethink your approach for storing the data.

[pcm314]

Hello @Cris LaPierre

Thank you very much for your answer, the way I'm storing my data is not arbitrary. I followed official examples on how to import data to Diagnostic Feature Designer APP of the Predictive Maintenace toolbox.

Import and Visualize Ensemble Data in Diagnostic Feature Designer - MATLAB & Simulink - MathWorks España

But following what you have said, I do have assigned the same variable type on the entire column, first three columns are timetables and the fourth is double.

It still intrigues me why creating the table as

table()

works fine, obiously it doesn't allow me to change my variable names, but it stores the data correctly.

[Cris LaPierre]

Follow up - there is one exception. Change the datatype of your table columns from timetable to cell. Then store each timetable as a cell.

sz = [3,4];
varTypes = {'cell','cell','cell','double'};
DataTable = table('size',sz,'VariableNames',{'channel1','channel2','channel3','ConditionVar'},'VariableTypes',varTypes)

DataTable = 3×4 table
      channel1        channel2        channel3      ConditionVar
    ____________    ____________    ____________    ____________

    {0×0 double}    {0×0 double}    {0×0 double}         0      
    {0×0 double}    {0×0 double}    {0×0 double}         0      
    {0×0 double}    {0×0 double}    {0×0 double}         0      

for i=1:height(DataTable)
    ConditionVar = randi([0 1],1);

    dt = 0.05;
    channel1 = rand(10,4);
    channel2 = rand(10,4);
    channel3 = rand(10,4);

    ch1 = array2timetable(channel1,'SampleRate',1/dt);
    ch2 = array2timetable(channel2,'SampleRate',1/dt);
    ch3 = array2timetable(channel3,'SampleRate',1/dt);

    DataTable.channel1(i) = {ch1};
    DataTable.channel2(i) = {ch2};
    DataTable.channel3(i) = {ch3};
    DataTable.ConditionVar(i) = ConditionVar;

end
DataTable

DataTable = 3×4 table
        channel1            channel2            channel3        ConditionVar
    ________________    ________________    ________________    ____________

    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         0      
    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         0      
    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         1      

[pcm314]

Thank you very much @Cris LaPierre, this workaround worked for me.

[Cris LaPierre]

This is the same approach used to create that table. If you look at the summary, you will see Vibration and Tacho are of data type cell.

load dfd_Tutorial dataTable
summary(dataTable)

Variables:

    Vibration: 16×1 cell

    Tacho: 16×1 cell

    faultCode: 16×1 double

        Values:

            Min          0    
            Median       1    
            Max          1    

[dpb]

But that certainly isn't the way the page at the given link represents/presents it --

View this table in your MATLAB® command window.

dataTable =

  16×3 table

        Vibration               Tacho           faultCode
    __________________    __________________    _________

    [6000×1 timetable]    [6000×1 timetable]        0    
    [6000×1 timetable]    [6000×1 timetable]        1    
    [6000×1 timetable]    [6000×1 timetable]        1    
    [6000×1 timetable]    [6000×1 timetable]        1    
    [6000×1 timetable]    [6000×1 timetable]        1    

Your example above does show each as a cell as signified by the curlies around the elements

    DataTable = 3×4 table
        channel1            channel2            channel3        ConditionVar
    ________________    ________________    ________________    ____________

    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         0      
    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         0      
    {10×4 timetable}    {10×4 timetable}    {10×4 timetable}         1      

but that is NOT the same, precisely as the example.

What's the difference; how was the example created/displayed as such?

Something definitely going on there that is unexplained as yet.

[Stephen23]

@dpb: the documentation example is impossible.

S = load('dfd_Tutorial')

S = struct with fields:
    dataTable: [16×3 table]

S.dataTable

ans = 16×3 table
        Vibration               Tacho           faultCode
    __________________    __________________    _________

    {6000×1 timetable}    {6000×1 timetable}        0    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        0    
    {6000×1 timetable}    {6000×1 timetable}        0    
    {6000×1 timetable}    {6000×1 timetable}        0    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    
    {6000×1 timetable}    {6000×1 timetable}        1    

[dpb]

That's what I thought was so; not used to TMW making such gaffs-- wonder how they managed to get such a foopah on the web page. It surely should be corrected to reflect reality.

[Cris LaPierre]

I agree. The square brackets threw me off originally as well. I will make a suggestion internally to the relevant team to update the doc page.

[dpb]

Thanks, Chris. That would likely save a bunch of other head-scratching by later arrivals, as well.

Which raises a Q? in my mind --

What's the point in the VarType parameter in the table function containing

'table'        Table with no variables
'timetable'    Timetable with no variables and NaT for row times

as possible variable data types when you can't then replace the empty timetable with a real one as I tried thinking that must have been how the example was created if it was possible at all.

Otherwise, one gets the embedded "table in a table" which, I suppose, one might figure out some reasons for, but embedding a table or timetable in a table as a cell is still just a cell.

Is there something else possible that caused that or is it another case of somebody getting too carried away in writing the documentation?

Normally a table is created from the data that gets recognized automagically, so hadn't ever perused the particular input before this exercise.

[Cris LaPierre]

VarType lets you predefine the variable type for each variable. This can be helpful when you are particular, and the autodetection may not get you what you want. It is an optional setting, but only makes sense when predefining a table with variables (i.e. with 'Size' name-value pair.

I am not sure you should be able to define a table with columns of VarType Table or Timetable. That seems like another area of potential improvement to me.

[dpb]

I understand the point in VarType, yes, when/if auto-recognition fails or you think it might fail or you want something like a categorical that would auto-detect as cellstr, say.

The point I was asking about is, indeed, specifically, the "why" of including Table or Timetable in the allowable list if there were some way to actually make use of it -- which the experiments I tried for the OP here to try to figure out how to create such an animal (as we thought we were after owing to the bum steer of the [] instead of {}) seemed to indicate that while one can create the table with the definition, it's not possible to use it to hold anything after having defined it so there seems no point.

Clarifying/removing/whatever is the right thing would, indeed, be an improvement.

[Cris LaPierre]

I agree. I cannot think of a scenario where it would work. In the feedback, I have also suggested they consider not allowing these data types for VarType.

[dpb]

Feedback appreciated, muchly...inquiring minds and all that... :)