lsqnon - take larger steps

Dear all,

I'm trying to use lsqnonlin to minimize the difference between the output of a model and experiments, in order to determine the best set of three parameters to use in the model. The problem is, whatever the initial point, lsqnonlin always converges in a few iteration, but it does not really moves from the initial point. I have investigated the solution field, and I see there is a clear minimum. I have played around with TolX and TolFun, but with no success. The parameters are in the order of magnitude of 1. If a record the parameters searched by lsqnonlin, it moves from the initial point by 1e-6 or even lower steps, and this is by far too strict for the solution I require. Is there a way to increase this step to 1e-2 or 1e-1?

Would other function better suit this problem (e.g. fminunc)?

thanks in advance Mauri

------------------------------------------------

I tried to make it easier. I fix all the parameters except for one to reasonable values, I apply a brute force method (I pick up 4 values of the parameters under fitting), and I see a minimum. Now, if I give lsqnon the worst initial point between those I investigated, it does not move toward the best value, it just does not move from there.

What could it be?

Answer 1

[Gabo]

It would be useful to see some code. From the description, though, you might be passing the wrong function to lsqnonlin.

Say the parameters you are fitting are a vector a, and your errors are err(a) = (err1(a),...,errN(a)). It seems that you are passing f(a) = err1(a)^2+...+errN(a)^2 to lsqnonlin. That would be incorrect. You need to pass err(a) to lsqnonlin, which internally squares the errors and adds them up. See the "Description" in the lsqnonlin help.

Alternatively, you can use fminunc, as you pointed out, which, indeed, takes f(a) as input.

Answer 2

[maurizio]

Thanks Gabo,

here some lines of code:

options = optimset('Display','iter','TolX',1e-6,'TolFun',1e-6); %I have tried also 1e-3

X0 = [5 7.5000 0.900];

lowerBounds = [0.1 2 0.8]; upperBounds = [10 20 1.2]; x = lsqnonlin(@mySimulat, X0,lowerBounds , upperBounds, options);

%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%

function diff = mySimulat(x)

%run the model with the parameters x; %output from the model are 50 points to be compared with experiments:

diffe(1:50,1) = Yexp(1:50,1)-Ysim(1:50); diff = diffe';

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

when I run it, I obtain:

Iteration................. 0 Func-count.................4 f(x).......................0.230268 Norm of step...............blank First-order optimality.....1.8e+004 CG-iterations..............blank

Iteration................. 1 Func-count.................8 f(x).......................0.230268 Norm of step...............7.9242e-007 First-order optimality.....1.8e+004 CG-iterations..............0

Iteration................. 2 Func-count.................12 f(x).......................0.230268 Norm of step...............1.98105e-007 First-order optimality.....1.8e+004 CG-iterations..............0

Optimization terminated: norm of the current step is less than OPTIONS.TolX.

x =[ 5.000000000000000 7.500000000000000 0.900000000000000]

Answer 3

[Gabo]

I see you were passing the right function. From your code, the only suspicious point is the last line of mySimulat. Is diffe a 50 by 1 vector or is it larger? The issue is, if diffe has more than 50 entries, diff also has more than 50 entries, and entries 51:end won't change. What happens if you set diff=Yexp-Ysim directly?

You can also try fminunc (as you mentioned before), to compare.

Answer 4

[maurizio]

I tried fminunc, it takes slightly larger steps (it changes the initial condition by 1e-3 instead of 1e-6), but still, whatever the initial conditions, it keeps on searching really close to it. I will investigate in the next days the shape of the solution field, to see if there are many local minima, up to now I just look into some points far away from each other. But I think it's strange that it happens with every initial point.

 I would just like to tell the solver not to take steps smaller than 1e-2, but I couldn't find any documentation.

If you have further suggestions I will try them.

regards

Answer 5

[Gabo]

Maurizio, unfortunately, it's not possible to force the algorithm to take a step of a minimum length --by design, the algorithm computes the step taking many things into consideration.

There could be something special about that point, or about the sensitivity of the simulation values to small changes around that point. Regarding the former, you can try different starting points. Regarding the latter, you may try changing the parameters that control the finite difference derivative estimation internally. For example, if the simulation is barely sensitive to small changes in the parameters, but it changes more noticeably for larger changes in the parameters, you can reset the DiffMinChange option to something like 1e-2, or 1e-4.