Is the Trust-Region-Reflective algorithm the same for minimizing a scalar function and solving a nonlinear system of equations?

I'm tring to solve a system of "blackbox" residual function (F(x)=0) and was wondering if it is better to solve the system or to minimize the sum of the equations as a single scalar function.

It seems the same if the system is solved as a least square problem using lsqnonlin, especially when looking to the available algorithms, like the Trust-Region.

Is the Trust-Region algorithm the same for minimizing a scalar function and solving a nonlinear system of equations?

[Umar]

Hi Marcus,

Please see my response to your comments.

I'm tring to solve a system of "blackbox" residual function (F(x)=0) and was wondering if it is better to solve the system or to minimize the sum of the equations as a single scalar function.

For well defined system of equations and well behaved smooth residuals, I will recommend solving system directly. On the other hand, residuals being noisy, non smooth or ill conditioned system, using least squares approach sounds robust.

It seems the same if the system is solved as a least square problem using lsqnonlin, especially when looking to the available algorithms, like the Trust-Region.

It can handle both scenarios.

Is the Trust-Region algorithm the same for minimizing a scalar function and solving a nonlinear system of equations?

It can adapt to both situations.

[Francisco J. Triveno Vargas]

Marcus I try to summarize my answer I use for fmincon and lsqnonlin to solve nonlinear system of equations with restrictions or not, options for both can be set as:

options = optimoptions(@lsqnonlin,'Algorithm','trust-region-reflective');

options = optimoptions('fmincon','Algorithm','trust-region-reflective',...

'SpecifyObjectiveGradient',true,'HessianFcn','objective');

Otherwise, fminsearch for example uses the simplex search methodand generally is for non-constraint problems.

In other words depends of your black box and the objectives that you follow.

[Marcus]

@Umar I share the same thoughs. In my problem, there are 2 variables and 2 residuals are individually nearly linear, which surfaces approaching planes, and smooth. Thus, it seems better to use the Newton-Raphson because it approximates the residuals by a linear model. But the documentation of fsolve says that the algorithm is the Trust-Region. As far as I know, this algorithm solves a scalar function aproximating it by a quadractic. So, fsolve never solves the system directly, it is the same with lsqnonlin.

Is that correct?

[Umar]

That is correct, @Marcus. Both fsolve and lsqnonlin indirectly solve systems and providing accurate solutions for nonlinear problems. However, Francisco suggested some good points as well to take them into consideration.

[J. Alex Lee]

The meanings of zero-ing ("solving") and minimizing are different, so it depends what you are looking for.

If your number of equations and unknowns match, you might be able to "solve" - if equations are nonlinear (would have to assume if black box), no guarantee.

If you have more unknowns than equations or vice versa, you cannot "solve" in general.

Since you say you have 2 equations and 2 unknowns, you can try solving, but a solution is not guaranteed. Not sure what "nearly linear" means, but maybe you can expect a solution, so I would see if fsolve works.

As for algorithms in general, they are related in that you can pose a minimizing problem as zero-ing the gradient. But not the same when you want to apply them to your particular problem

As for fsolve in particular and trust region - I never really understood exactly how it works either, but I think it should be based on Newton at its core with a lot of frills to make it robust against the general case - in any case I trust that since fsolve purports to solve systems of equations, it will [attempt to] do so whatever its internal algorithm is based on; it won't minimize.

If you have an idea of the neighborhood of values of unknowns to expect a zero, you can maybe visually map out the contour plot of each residual w.r.t. a grid of values of unknowns to find the zero-contour, and see if they appear to coincide anywhere?