Overwriting table columns with ()-indexing

Suppose I have two tables,

T1=array2table(rand(5,3))

T1 = 5×3 table
       Var1         Var2        Var3  
    __________    ________    ________

       0.87688    0.016374    0.091275
       0.44249     0.98478     0.14807
       0.84115     0.28061    0.044424
    0.00060432     0.91388     0.32333
       0.85832     0.49634     0.87334

T2=array2table(char(randi([65,90],5,3)),Var={'Col1','Col2','Col3'})

T2 = 5×3 table
    Col1    Col2    Col3
    ____    ____    ____

     Q       W       N  
     T       N       J  
     D       J       W  
     V       D       Y  
     C       A       P  

I wish to replace the final column of T1 with that of T2. One way to accomplish this is,

T3=[T1(:,1:2) , T2(:,3) ]  

T3 = 5×3 table
       Var1         Var2      Col3
    __________    ________    ____

       0.87688    0.016374     N  
       0.44249     0.98478     J  
       0.84115     0.28061     W  
    0.00060432     0.91388     Y  
       0.85832     0.49634     P  

Why, though, does this indexing approach not give the same result?

T3=T1;
T3(:,3)=T2(:,3)

T3 = 5×3 table
       Var1         Var2      Var3
    __________    ________    ____

       0.87688    0.016374     78 
       0.44249     0.98478     74 
       0.84115     0.28061     87 
    0.00060432     0.91388     89 
       0.85832     0.49634     80 

The result is the same as what would be obtained with brace indexing,

T3=T1;
T3{:,3}=T2{:,3}

T3 = 5×3 table
       Var1         Var2      Var3
    __________    ________    ____

       0.87688    0.016374     78 
       0.44249     0.98478     74 
       0.84115     0.28061     87 
    0.00060432     0.91388     89 
       0.85832     0.49634     80 

I understand why brace-indexing gives this result, but I thought ()-indexing was supposed to keep the data contained in a table. Why don't the tabular properties of T2(:,3), like the column name and data type, get transfered as well when parentheses are used?

[Stephen23]

I think the table subsind/subsref operations are much more overloaded than they first appear to be. One might expect them to have a similar nature array indexing, i.e. that they refer to actual locations in memory (of pointers or data). But given that each column/variable seems to be stored as its own array in memory, implementing generalized parenthesis indexing over multiple columns/variables is non-trivial. And I suspect, somewhat overloaded.

We can already see this in the fact that one can assign cell arrays to a table:

T = array2table(rand(3,5))

T = 3×5 table
     Var1       Var2       Var3       Var4       Var5  
    _______    _______    _______    _______    _______

    0.86481    0.82302    0.52844      0.315    0.50188
    0.83843    0.36024    0.44834    0.59645    0.40722
    0.68251    0.19563    0.65567    0.11702    0.05087

T(2,3:5) = {0,42,99}

T = 3×5 table
     Var1       Var2       Var3       Var4       Var5  
    _______    _______    _______    _______    _______

    0.86481    0.82302    0.52844      0.315    0.50188
    0.83843    0.36024          0         42         99
    0.68251    0.19563    0.65567    0.11702    0.05087

Answer 1

[Stephen23]

I supect that the cause is consistency of behavior. Consider your example but not replacing the entire column, but only from one row:

T3(1,3) = T2(1,3)

What would be the expected behavior? It seems reasonable to retain the existing class of the LHS. If this is the design decision, then replacing any number of rows retains the LHS class, otherwise we get an inconsistent behavior that depends on the sizes of the two tables:

 # rows  | output
replaced | class
---------|-------
all - N  | LHS
...      |
all - 3  | LHS
all - 2  | LHS
all - 1  | LHS
all - 0  | RHS ! ouch !

Consider that we write some code that replaces x-rows of a table (double) with y-rows of new (uint8) data.. which works perfectly (the column/variable class of the remains double) until one day x==y. Then because the column silently changes type, and our code's behavior completely changes... ouch.

[Matt J]

Consider your example but not replacing the entire column, but only from one row:...What would be the expected behavior?

One might expect it to throw an error, with the reasoning that, if the desire is to replace the data in one row, you should use {}-indexing.

But, I suspect you're right about the rationale.

[Stephen23]

"One might expect it to throw an error, with the reasoning that, if the desire is to replace the data in one row, you should use {}-indexing."

That would restrict users to only allocating homogenous data, so would exclude assigning a table or cell array with heterogenous types. Which would be quite restrictive, given the target use-cases of tables.

[Matt J]

That would restrict users to only allocating homogenous data

How so? {}-indexing can assign heterogeneous data.

Z=["A","B","C","D"]';

T=[array2table(rand(4,3)), table(Z)]

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.56267    0.09383    0.97628    "A"
    0.83868    0.44406    0.82385    "B"
    0.89272    0.49562    0.58937    "C"
    0.11445     0.5021     0.1391    "D"

T{end,:}=T{1,:}

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.56267    0.09383    0.97628    "A"
    0.83868    0.44406    0.82385    "B"
    0.89272    0.49562    0.58937    "C"
    0.56267    0.09383    0.97628    "A"

[Stephen23]

"How so? {}-indexing can assign heterogeneous data."

No, it can't. Your intermediate data are all converted to string (following the basic rules for concatenation of any data with string type), because {} on a table concatenates all of the data together:

Z = ["A";"B";"C";"D"];
T = [array2table(rand(4,3)), table(Z)]

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.33831    0.65564    0.25738    "A"
    0.16718    0.32565    0.90085    "B"
     0.1289    0.19723    0.65919    "C"
    0.44084    0.38078    0.85909    "D"

T{1,:} % concatenated -> string array

ans = 1×4 string array
    "0.33831"    "0.65564"    "0.25738"    "A"

So your example is a lossy conversion involving two type conversions. Lets check the loss of data:

T{2,:} = T{1,:} % lossy numeric -> string -> numeric

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.33831    0.65564    0.25738    "A"
    0.33831    0.65564    0.25738    "A"
     0.1289    0.19723    0.65919    "C"
    0.44084    0.38078    0.85909    "D"

T{2,1:3} - T{1,1:3} % numeric only

ans = 1×3

1.0e-05 * -0.4645 0.1444 0.3416

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Two superfluous type conversions and data that retains only five significant figures... no thanks.

[Paul]

Heterogeneous row data can be assigned using ()-indexing on the LHS and a cell array on the RHS. There seems to be an undocumented capability here.

Z=["A","B","C","D"]';
T=[array2table(rand(4,3)), table(Z)]

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.39006     0.3498    0.26884    "A"
    0.55866    0.33409    0.36504    "B"
     0.3273    0.95403    0.81165    "C"
    0.85734    0.10814    0.28922    "D"

"One way to assign or add a row to a table is to assign a cell array to a row. If the cell array is a row vector and its elements match the data types of their respective variables, then the assignment converts the cell array to a table row. However, you can assign only one row at a time using cell arrays. "

T(2,:) = {T{1,1},T{1,2},T{1,3},T{1,4}}  % assign a single row, as advertised

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.39006     0.3498    0.26884    "A"
    0.39006     0.3498    0.26884    "A"
     0.3273    0.95403    0.81165    "C"
    0.85734    0.10814    0.28922    "D"

T{2,1:3} - T{1,1:3}

ans = 1×3

0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

T(3:4,:) = repmat({T{1,1},T{1,2},T{1,3},T{1,4}},2,1)  % assign multiple rows, undocumented

T = 4×4 table
     Var1       Var2      Var3       Z 
    _______    ______    _______    ___

    0.39006    0.3498    0.26884    "A"
    0.39006    0.3498    0.26884    "A"
    0.39006    0.3498    0.26884    "A"
    0.39006    0.3498    0.26884    "A"

T{2:4,1:3} - T{1,1:3}

ans = 3×3

0 0 0 0 0 0 0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

The doc has a nice page on table indexing for accessing data in a table. Would be nice if it had a companion page for table indexing on the LHS of an assignment statement (all I can find are examples).

[Matt J]

OK, well that clears up a fair bit. It seems to me, though that what would be desirable is a 3rd kind of indexing delimiter, equivalent to table2cell. Then you could reserve ()-indexing for container-preserving assignments.

Z=["A","B","C","D"]';
T=[array2table(rand(4,3)), table(Z)]

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.13771     0.4636    0.54469    "A"
    0.59651    0.14219    0.99137    "B"
    0.88095    0.81535    0.20977    "C"
    0.66864    0.67392    0.12098    "D"

T(3:4,:)=table2cell(T(1:2,:)) %Would be nice to replace this by T(3:4,:)=T[1:2,:] 

T = 4×4 table
     Var1       Var2       Var3       Z 
    _______    _______    _______    ___

    0.13771     0.4636    0.54469    "A"
    0.59651    0.14219    0.99137    "B"
    0.13771     0.4636    0.54469    "A"
    0.59651    0.14219    0.99137    "B"

[Paul]

"If the cell array is a row vector and [the data types of] its elements match the data types of their respective variables, then the assignment converts the cell array to a table row."

Z=["A","B","C","D"]';
T=[array2table(rand(4,3)), table(Z)]

T = 4×4 table
     Var1       Var2        Var3        Z 
    _______    _______    _________    ___

    0.28335     0.6523      0.83174    "A"
    0.16001    0.91971    0.0033429    "B"
    0.85625    0.72206      0.52888    "C"
    0.24957    0.54239       0.4097    "D"

Apparently the meaning of "match" is a bit loose in this context.

T.Properties.VariableTypes

ans = 1×4 string array
    "double"    "double"    "double"    "string"

T(1,:) = {uint8(1),uint32(2),uint64(3),'z'}

T = 4×4 table
     Var1       Var2        Var3        Z 
    _______    _______    _________    ___

          1          2            3    "z"
    0.16001    0.91971    0.0033429    "B"
    0.85625    0.72206      0.52888    "C"
    0.24957    0.54239       0.4097    "D"

But then I don't understand why this doesn't work insofar as char is convertible to double

T(2,:) = {'w','x','y','z'}

Error using () (line 120)
Conversion to double from cell is not possible.

And I don't understand the error message. Where exactly is the attempt to convert from a cell to double? I'd think the conversions apply to contents of the cell on the RHS, not to the cell itself.

Answer 2

[Srinjoy]

T3(:,3) = T2(:,3);

When you do the above,

MATLAB is not actually replacing the third variable of T3. Instead, it is modifying the existing variable Var3.

Since Var3 in T3 is already of type double, MATLAB keeps that type and tries to fit the new data into it. The column from T2 contains characters, so MATLAB converts those characters to their numeric codes. That is why you see values like 78, 74, etc., which correspond to the ASCII values of 'N', 'J', and so on.

But, when you do

T3 = [T1(:,1:2), T2(:,3)];

you are creating a completely new table. In this case, MATLAB takes the third variable directly from T2, including its type and name, so it remains a character column.

So even though parentheses indexing returns a table, assignment with () does not necessarily copy over the variable metadata. If the target variable already exists, MATLAB preserves its type and assigns the new values into it, performing any required type conversion.

Hope this helps.

[Matt J]

Yes, I know that is what it is doing. The question was why? Shouldn't {}-indexing and ()-indexing have distinct assignment behavior?

Answer 3

[Matt J]

Apparently, complete replacement of table columns can't be done via direct subsasgn indexing, so here is a function that will do it.

T1=array2table(rand(5,3))

T1 = 5×3 table
     Var1       Var2        Var3   
    _______    _______    _________

    0.76666    0.64845    0.0011167
    0.38064    0.24385      0.83112
     0.1378    0.39644      0.72924
    0.65116    0.14977     0.012019
      0.113    0.20396     0.084029

T2=array2table(char(randi([65,90],5,3)),Var={'Col1','Col2','Col3'})

T2 = 5×3 table
    Col1    Col2    Col3
    ____    ____    ____

     B       M       Y  
     U       B       T  
     U       V       O  
     X       O       W  
     U       J       P  

T3=replacevars(T1,3,T2(:,3))

T3 = 5×3 table
     Var1       Var2      Col3
    _______    _______    ____

    0.76666    0.64845     Y  
    0.38064    0.24385     T  
     0.1378    0.39644     O  
    0.65116    0.14977     W  
      0.113    0.20396     P  

[Paul]

I had to recalibrate on where this question started ... which was:

"I wish to replace the final column of T1 with that of T2. "

Why isn't ordinary .field indexing sufficient to achieve the desired assignment?

rng(100);
T1 = array2table(rand(5,3))

T1 = 5×3 table
      Var1        Var2       Var3  
    _________    _______    _______

       0.5434    0.12157    0.89132
      0.27837    0.67075     0.2092
      0.42452    0.82585    0.18533
      0.84478    0.13671    0.10838
    0.0047189    0.57509     0.2197

T2 = array2table(char(randi([65,90],5,3)),Var={'Col1','Col2','Col3'})

T2 = 5×3 table
    Col1    Col2    Col3
    ____    ____    ____

     Z       L       J  
     V       Y       A  
     E       V       G  
     V       I       U  
     H       E       A  

T3 = T1;
T3.Var3 = T2.Col3

T3 = 5×3 table
      Var1        Var2      Var3
    _________    _______    ____

       0.5434    0.12157     J  
      0.27837    0.67075     A  
      0.42452    0.82585     G  
      0.84478    0.13671     U  
    0.0047189    0.57509     A  

[Matt J]

Why isn't ordinary .field indexing sufficient to achieve the desired assignment?

Because only the data is copied over. Properties like the variable names and units are not transferred.

rng(100);

T1 = array2table(rand(5,3));
 T1.Properties.VariableUnits={'mm','mm','mm'};
 
T2 = array2table(char(randi([65,90],5,3)),Var={'Col1','Col2','Col3'});

T3 = T1; 
  T3.Var3 = T2.Col3;

T3.Properties.VariableNames

ans = 1×3 cell array
    {'Var1'}    {'Var2'}    {'Var3'}

T3.Properties.VariableUnits

ans = 1×3 cell array
    {'mm'}    {'mm'}    {'mm'}

Conversely,

T4=replacevars(T1,3,T2(:,3));

T4.Properties.VariableNames

ans = 1×3 cell array
    {'Var1'}    {'Var2'}    {'Col3'}

T4.Properties.VariableUnits

ans = 1×3 cell array
    {'mm'}    {'mm'}    {0×0 char}