starting vector (zero vector) equals lower bounds but gets projected to non-zero vector

I created a small example where I created a start vector euqal to the lower bounds, so the start vector respects the bounds, thought gets projected to non-zero vector when double-checking inside objective function.

Is this a bug or do I miss something here?

n = 5;

lb = zeros(n,1);
ub = Inf(5, 1);

startVec = zeros(n, 1);

sol =  fmincon(@(x)func(x), startVec, [], [], [], [], lb, ub);

function fval = func(x)
% start vector (zero vector) becomes [0.99 0.99 0.99 0.99 0.99]
if any(x ~= 0)
    error('Unexpected values: x is not the zero vector. Current x: %s', num2str(x'));
end
end

[Bruno Luong]

Why it's a bug? The minizer might explore anywhere it wants to.

Also don't understand your test any(x ~= 0)

[SA-W]

If the docu states that x(i) >= lb(i), I expect that a start vector satisfying these bounds does not get projected to some nonintutive values.

[Bruno Luong]

It does what it does, user should not want to interfer with the optimizer while it is working. Only the final end result it returns count.

Pratically any numerical floating point comparison implementation outthere work with some sort of tolerance, each decides the tolerance to be resonable (based on the estimate scale of your data) in practice. The scale estimation is often empirical, and more like an art than precice math, we just have to accept it.

So far your question does not show anything wrong or bug with FMINCON as far as I can see it.

[Bruno Luong]

More interesting observation is that the there is always a strict positive tolerance to the constraints on interior point algorithm. Code based on Matt's demo show that in the final solution

n = 5;
lb = zeros(n,1);
ub = Inf(n,1);
startVec = ones(n,1);

opts = optimoptions('fmincon','Algorithm','sqp');
sol = fmincon(@func, startVec, [], [], [], [], lb, ub, [], opts)

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

sol = 5×1

0 0 0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

opts = optimoptions('fmincon','Algorithm','interior-point');
sol = fmincon(@func, startVec, [], [], [], [], lb, ub, [], opts)

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

sol = 5×1

1.0e-07 * 0.5003 0.5003 0.5003 0.5003 0.5003

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

function fval = func(x)
fval = sum((x+1).^2);
end

Answer 1

[Walter Roberson]

Although the documentation says that lb specifies that x(i) >= lb(i) for all i the implementing code has

        violatedLowerBnds_idx = XOUT(xIndices.finiteLb) <= l(xIndices.finiteLb);

and when true, shifts the bounds away from the starting point.

Notice the <= in the test -- so an input vector that is exactly equal to the lower bounds is considered to be in violation of the bounds.

This is arguably a bug in the implementation.

[SA-W]

So when I want to model a start vector equal to the lower bounds vector, I would do something like

startVec = zeros(n, 1) + eps; % ?

[Walter Roberson]

startVec = lb + eps;

[Walter Roberson]

startVec = lb + eps(lb);

would arguably be better.

[SA-W]

How to control the eps-shift at intermediate iterations? Suppose the true solution is equal to the lb vector, i.e. the bounds are active. The solver does not allow that and shifts all x(i) where x(i) = lb(i) away.

[Walter Roberson]

You just have to live with it.

You can file a bug report with Mathworks that strictly equal to lower bound is being rejected.

[Bruno Luong]

Why and when it is rejected? What is SOL and exitflag output of FMINCON after let it does the thing freely rather than stop it?

Self checking the constraints in the cost evaluation function then intercept and throw an error is nonsense intervention IMO.

Do it on SOL, not when FMINCON is working. We usually don't understand why the optimizer needs to evauate the model at some specific point, even it looks odd at the first look, eg it needs to evaluate the gradient or such.

[SA-W]

For default solver options, the first point to be evaluated is the start point. Changing "startVec = lb;" to "startVec = lb + eps(lb);", the start vector is indeed passed without modification to the objective function and is the first point seen by the objective above. This is not the case for "startVec = lb;" where the start vector gets projected. I do not want to assess whether this does not make sense or not, but it clearly does not align with the documentation of fmincon and friends.

[Bruno Luong]

"the first point to be evaluated is the start point"

Where is this description?

The fmincon doc says quote

"x0 — Initial point

'interior-point' algorithm — If the HonorBounds option is true (default), fmincon resets x0 components that are on or outside bounds lb or ub to values strictly between the bounds."

In the example you posted above, your x0 is on lb so it resets strictly between the bounds. It does look like it does exactly what in this description. I don't see anything wrong.

It is not clear why you want to control what FMINCON does.

Write your own optimiser if you want to control everything.

[SA-W]

I am happy with what fmincon does. No need to reinvent the wheel.

Just wanted to clarify if the observation I made is the intented behavior of fmincon because it looked odd to me.

@Walter Roberson Maybe you want to rethink your answer?

[Walter Roberson]

Why would I want to rethink my answer??

Answer 2

[Matt J]

This behavior is specific to the interior-point algorithm. As the name suggests, an interior-point algorithm must start at an interior point.

Demo('sqp')

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

sol = 5×1

1.0000 1.0000 1.0000 1.0000 1.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Demo('interior-point')

Error using solution>Demo/func (line 19)
Unexpected values: x is not the zero vector. Current x: 0.99 0.99 0.99 0.99 0.99

Error in objfunEvaluator (line 5)
fval = feval(Objfun, x, self.FunArgs.AdditionalParameters{:});

Error in OptimFunctions/objectiveFirstEval (line 645)
[fval, grad, hess] = self.ObjectiveFunAndGrad(self,self.FunFcn{3},...

Error in fmincon (line 501)
[initVals.f,initVals.g,HESSIAN,funObj] = funObj.objectiveFirstEval(X);

Error in solution>Demo (line 13)
sol = fmincon(@func, startVec, [], [], [], [], lb, ub,[],opts)

Caused by:
Failure in initial objective function evaluation. FMINCON cannot continue.

function Demo(alg)

n = 5;
lb = zeros(n,1);
ub = Inf(5, 1);
startVec = zeros(n, 1);

FirstCall=true; 
opts=optimoptions('fmincon','Algorithm',alg);
sol = fmincon(@func, startVec, [], [], [], [], lb, ub,[],opts)


    function fval = func(x)

        if any(x ~= 0) && FirstCall
            error('Unexpected values: x is not the zero vector. Current x: %s', num2str(x'));
        else
            fval=norm(x-1)^2; FirstCall=false;
        end
        
    end
end

[Bruno Luong]

Yes exactly, it's even describeb in the doc where I hightlighted the relevant paragraphe here

[Matt J]

I updated with a more PC-friendly demo.