How to perform double summation using lsqnonlin

I understand that lsqnonlin does the sum of squares of a vector defined. BUt how exactly does it perform a double summation?

I am trying to obtain the min for two 3D vectors. the vector defined isn't getting optmized as I would expect.

[Askic V]

You could start by providing code with example and expected result. There is a better chance to get help that way.

[Akella Kartik]

There is no proper code as such. This is more of a generic query so that I can code further.

There are two 3D vectors, say of sizes (m,2,n). And I want to compute the norm using the above equation- for each 2 dim vector at each position in (m,2) arrays across the n dimensions.\

I am trying to

X0 = some initialiazation;

[x,resnorm] = lsqnonlin(@ComputeError,X0);

The logic in the function ComputeError would be as below:

i = 1:n

error = norm(A(:,:,i) - B(:,:,i) ); %A and B are some vectors computed using the X0

But the above code is squaring the sum of norms across (m,2) vectors for n dimensions. WHile, what I ideally want is the sum of squares of the norms, summed over the (m,2) across n.

[Askic V]

Ok, so you need to minimize a cost function defined in the following way:

function cost = cost_func(x)
    m = 5;
    n = 3;
    A(:,:,1:n)= rand(m, 2, n);
    B(:,:,1:n) = rand(m, 2, n);
    
    % some processing with x, A,B
    % x is a vector 2x1
    cost = 0;
    for i = 1:n
        for j = 1:m
            cost = cost + norm(A(j,:,i)-B(j,:,i))^2;
        end
    end
end

[Akella Kartik]

But when calling the lsqnonlin, it should do the sum of squares, isn't it?

In your code, the squaring and summation is done explicitly. This works for me though, but I wonder what happens if I use call the lsqnonlin.

Thanks in advance!

[Torsten]

For lsqnonlin:

function f = cost_func(x)
  for i = 1:n
    for j = 1:m
        f((i-1)*m + j) = A(j,:,i)-B(j,:,i);
    end
  end
end

[Askic V]

@Torsten, can you please elaborate this part?

f((i-1)*m + j) = A(j,:,i)-B(j,:,i);

[Torsten]

lsqnonlin minimizes sum_i f_i^2 where the f_i are (nonlinear) functions of a parameter vector x to be optimized. Since A(j,:,i) and B(j,:,i) are vectors, my previous answer will let MATLAB throw an error.

But instead, simply do

function f = cost_func(x)
  A = ...;
  B = ...;
  f = A(:)-B(:);
end

[Askic V]

Now, it is more clear to me. Thank you.

[Akella Kartik]

Thanks @Askic V/ @Torsten for the responses!

f = A(:)-B(:) is ideally computing the differences of the vectors and the lsqnonlin would min the squared sums. But, going by the equation I have posted, shouldn't the norm of these two be explicitly defined?

Also, f((i-1)*m + j) = A(j,:,i)-B(j,:,i) seemed logical with the indices stated for f.

If we were to take the norm, would this is be okay?

f((i-1)*m + j) = norm(A(j,:,i)-B(j,:,i))

This is not throwing up any errors, but has been running for more than an hour now. Values though are being updated quite well.

[Torsten]

Both settings

f = A(:)-B(:) 

and

f((i-1)*m + j) = norm(A(j,:,i)-B(j,:,i))

come out the same for lsqnonlin.

lsqnonlin squares norm(A(j,:,i)-B(j,:,i)) for all i and j and sums them.

But

sum_ij (norm(A(j,:,i)-B(j,:,i)))^2 = sum_ijk (A(j,k,i)-B(j,k,i))^2 = sum (A(:)-B(:)).^2

Of course, if your "norm" is not the usual Euclidian norm, the results will be different.

[Akella Kartik]

My norm is the usual Euclidean norm.

[Torsten]

Then both formulations are equivalent - choose the one you like best.

I suggest to take

f = A(:)-B(:)

because there are no square roots in it when the solver takes derivatives with respect to the parameters.

[Akella Kartik]

Got it! Thanks a lot.

Apart from the above suggestion, is any other thing that I need to take care of, that can actually provide me with better results and a faster optmization?

[Torsten]

We don't know the underlying problem - so it's hard to say anything useful.