why getting infeasible solutions using fmincon? (Update)

Hi

I have an optimization problem with many optimization variables and constraints (~561 var). I tried to solve the problem with fmincon but it gives the exit flag -2, which means there is no feasible solution.

For this reason, I thought, maybe the constraints (linear and nonlinear) don't have any feasible solution. So, I tried to consider these constraints as a system of equations and solve it with 'fsolve' for the same initial guess that I used for fmincon. fortunately, fsolve leads to an acceptable answer which is very close to the supplied initial guess.

Then I updated the original problem(which is solved by fmincon) with this new initial guess. But still the exit flag is -2 (infeasible). Then I thought the problem must be for bounding of optimization variables (lb,ub). So, I relaxed them and got exit flag 4. I also, wrote linear constraint as A,B matrices for both fsolve and fmincon. So, Based on the above explanation, what is exit flag 4? does it means the problem is feasible and I should accept its answer?

Can I solve the constraints (as an equation system) for the given initial guess considering the bounding of variables with fsolve as well in order to really understand if it has a feasible solution within the bounding?

Your help and suggestions are much appreciated.

[Torsten]

Does fmincon also fail if you start with the solution obtained from fsolve (and which you consider feasible) ?

Best wishes

Torsten.

[Jamais avenir]

Thanks for replying @Torsten . Yes, moreover the initial guess that I supply to fmincon (or even fsolve) is the answer of these non linear constraints (system)too.

[Torsten]

Then I suggest you generate a feasible starting point (using fsolve) and continue with fmincon in the usual way.

Best wishes

Torsten.

[Jamais avenir]

hmm, I did that few minutes ago, but didn't work @Torsten

[Jamais avenir]

Dear @Torsten I edited my question and added my attempts with fmincon and fsolve function. If its convenient please check them.

Answer 1

[Brendan Hamm]

fmincon is a gradient based solver and as it is a discretized version of the optimization problem these are approximated by the solver (both the gradient and the hessian). You may look into supplying these values yourself if you can analytically solve them.

You can suplpy the gradient in the objective and nonlinear constraint functions and then turn the GradObj on using optimoptions:

optimoptions for fmincon

User Supplied Hessian

Furthermore the discretization allows for modest violations of the feasible region. By default this is set to 1e-6, you can change this by changing the TolCon option using optimoptions.

To really narrow in on your problem, one would need more information about the actual problem being solved. Just because there is a 0 from fsolve on your nonlinear equality constraints does not imply that this is a feasible solution given other constraints/bounds. My guess is this is likely your issue, the points you find are not satisfying bounds/equality constraints, or perhaps even return an Inf or NaN in your objective.

[Jamais avenir]

Dear Brendan. Thanks for your hints. but I couldn't solve it yet.I have neatly summarized the part of code for both fsolve and fmincon cases. May I send you the part of code that results in this issue, so that you can see the problem more precisely.

[Brendan Hamm]

Yes. Please just attach it using the paper-clip icon. I'll warn that I may not have the time to get to it ... or likely until Monday, but others on the site may be able to help as well.

[Jamais avenir]

Thanks Brendan. I edited my question and added my codes in separate files for both fsolve and fmincon cases . Please check it when you have time.

[Brendan Hamm]

Just looking at this real briefly, it appears you are not solving the same set of equations. In your fmincon you have 561 design variables, whereas with fsolve you have 484. Not to mention your objective function and nonlinear constraints are expecting different size inputs. If I try and call your nolconfun with your Dat.InitialGuess I get the error:

Index exceeds matrix dimensions.

Error in nolconfun (line 68) pji = X( 451:496 )';

as you try and index beyond the size of the initial guess. For this reason you will NOT have a feasible point. Your non-linear constraint function must take your vector of design variables as input and therefore they should have the same size.

[Jamais avenir]

Dear Brendan. I modified my question above and updated the attach file based on your comment. Now, both fmincon and fsolve use 561 variables. Please kindly check my problem.

Thanks.

Answer 2

[Matt J]

Based on the above explanation, what is exit flag 4

A full table explaining all of the exit flags is in the fmincon documentation here. Exitflag=4 is one of the stopping criteria which fmincon interprets as a potential success. Basically, fmincon has decided that the size of the step taken at the last iteration was insignificant as measured by the choice of the TolX parameter, and assumes this to be a sign of convergence.. Obviously, the criterion gets more stringent as you take TolX smaller and smaller. If you think you may not have selected a stringent enough TolX, it may help to test with still smaller TolX values and see if the solution remains stable.

On the surface, it looks like the problem simply wasn't feasible until you made the lb,ub bounds large enough.

[Jamais avenir]

Thanks for your answer Matt J.

I think the problem is with my constraints. But I don't know what is the problem with it. I have mathematical equations and also GAMS version of this problem already. I checked my code with those script but, I can't understand why my Matlab codes are wrong.

[Matt J]

If GAMS gives a solution that you trust, then it should be a simple matter to substitute that solution into each of your MATLAB-implemented constraints and see which constraints are violated. The violated constraints will be the ones you should check for bugs.

If no constraints are violated, then you can also use the GAMS solution as the initial guess in fmincon and see if fmincon diverges significantly from that initial point. If it doesn't diverge, it shows that fmincon agrees with GAMS that that point is a solution. The other solution fmincon was previously giving you may then have been either,

1. an additional solution
2. an inferior solution to GAMS, e.g., a local minimum that fmincon got stuck at.
3. a superior solution to GAMS, e.g., GAMS got stuck at a local minimum and you shouldn't be trusting it as much as you think.

All three cases above can be checked by looking at the objective function values of the different solutions.

[Jamais avenir]

Thanks for your answer Matt J.

I am not familiar with GAMS at all(it looks Gibberish to me), but I have the same problem there which yields to a solution. I think I must start to study GAMS and implement my problem there myself too.

Also, I want to know what bug do I have in my constraints so maybe I can stick to Matlab. As you said," substitute that solution into each of your MATLAB-implemented constraints and see which constraints are violated. " So How can I substitute GAMS solution to each of your MATLAB-implemented constraints?

Is it possible to trace GAMS code step bye step and see the outputs in something like workspace in Matlab as well?

[Matt J]

My previous comment was assuming you already had a solution of the problem that you trusted (from GAMS, or elsewhere). If you have a version of the problem where the solution is known, a good first test is to see whether the known solution x satisfies the constraints as you've provided them to MATLAB. Run x through your nonlcon function and apply the linear in/equalities A*x<=b etc... to see if the constraints are satisfied. Scrutinize the ones that are not.

A good second test is to see if fmincon thinks your solution is optimal. If you initialize fmincon with a correct solution, it should stop right away with that same solution.

On the other hand, if you don't have a version of the problem where the solution is known, it's not clear why are questioning the results you are now getting. The hard truth might be that a solution simply does not exist in some cases. You can't rely on intuition for everything!

[Jamais avenir]

Thanks again for your elaborated answer.

It is originally a multi-objective programming problem which is already solved by someone in GAMS with epsilon constraint method But I want to solve it with NBI (Non boundary intersection) method. In either method we need to calculate the Anchor points which are obtained by individually optimizing each objective function. At this point, I want to optimize these three objective functions that I have in this problem individually to obtain the anchor points. The solution is also know as it was calculated in GAMS already (calculation of Anchor point in epsilon constraint and NBI and etc is same thing). I am trying to work on it as you suggested in your comment.