gather() in tall array inverse is incorrect

Hi,

I am testing simple code with tall array inverse and gather(). The following results are different, anyone knows how to fix gather? As I am using tall array for AA and bb.

AA = ones(13,9);
bb = ones(13,1);
AA_t = tall(ones(13,9));
bb_t = tall(ones(13,1));

gather(AA_t\bb_t);

Unrecognized function or variable 'AA_t'.

AA \ bb;

[Stephen23]

Defining AA and bb consist of only 1's is ... not going to work, as the warning tells us:

AA = ones(13,9);
bb = ones(13,1);
gather(tall(AA)\tall(bb))

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.13 sec
Evaluation completed in 0.22 sec

ans = 9x1

1.0e+109 * -0.0000 0.0000 -0.0000 0.0000 -0.0000 0.0000 -0.0000 -2.7433 2.7433

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

AA \ bb

Warning: Rank deficient, rank = 1, tol = 1.040771e-14.

ans = 9x1

1.0000 0 0 0 0 0 0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Using values that are not rank deficient produces much more sensible results:

AA = rand(13,9);
bb = rand(13,1);
gather(tall(AA)\tall(bb))

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.022 sec
Evaluation completed in 0.053 sec

ans = 9x1

0.4724 0.3454 -0.3305 -0.1144 0.0596 0.2326 -0.0436 0.4004 -0.0722

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

AA \ bb

ans = 9x1

0.4724 0.3454 -0.3305 -0.1144 0.0596 0.2326 -0.0436 0.4004 -0.0722

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Lets try with the uploaded data:

S = load('AAbb.mat')

Warning: You must use MATLAB release 2023a to load this file. Alternatively, use WRITE(LOCATION,TT) to write a tall array to disk with version independent files.

Warning: You must use MATLAB release 2023a to load this file. Alternatively, use WRITE(LOCATION,TT) to write a tall array to disk with version independent files.

S = struct with fields:
      AA: [13x9 double]
    AA_t: [13x9 tall]
      bb: [13x1 double]
    bb_t: [13x1 tall]

S.AA \ S.bb

Warning: Rank deficient, rank = 7, tol = 1.040771e-14.

ans = 9x1

0.0195 0.0461 0.9691 0.8912 0.9073 0.9145 0 0 0.0034

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Note how the warning even tells us the rank of your matrix AA.

Apart from the values that you are using, it is unclear what the problem is.

[chen zhe]

So the problem still here: why gather（）cannot evaluate the same results as non-tall array case？ Is there a solution to make consistent results： A/b and gather（A_t/b_t）？ They all have the rank deficient issue

[Stephen23]

"why gather（）cannot evaluate the same results as non-tall array case?"

For efficient handling of the different data types they use different algorithms. No surprises there.

"Is there a solution to make consistent results： A/b and gather（A_t/b_t）？ They all have the rank deficient issue"

There are infinitely many solutions. Why should we expect any one solution to be preferred?

https://www.mathworks.com/matlabcentral/answers/562967-solution-to-ax-b

https://www.mathworks.com/matlabcentral/answers/260330-given-a-rank-deficient-design-matrix-fitlm-does-not-use-pinv-or-lsqr-to-find-the-minimum-norm-solut

Answer 1

[Aditya]

Hi Chen,

please modify this line and see whether the issue persists or not!

gather(AA_t)\gather(bb_t);

[chen zhe]

It works. But this is not what we want, as it uses two gather(). When we process tall arrays, we try to use gather() as less as possible due to time consuming.