fmincon and Getting "Solution" Before Convergence Tolerance is Hit

I'm running a relatively large and slow fitting routine through fmincon – there are 12 bounded parameters. As a result, on the available machines we have, the fit has taken several days, so far. I also set the tolerance to be way lower than we need, for our purposes. As a result, when the function output is plotted (to the Optimization PlotFcns window), the function output has effectively converged (each iteration does not change, to within what I need). Is there a way to stop the script and get the current "solution" (the current values fmincon has), without re-running the script?

I'm looking to save several days' worth of time here.

Answer 1

[Adam Danz]

From the Editor, press the pause button (first panel below). Execution will eventually pause but it may not be within the function that contains your call to fmincon() so you'll need to step through using the 'step' button in the 2nd panel below until you get to the fmincon outputs. You can save all of the existing variables so you can later reload them and start where you left off.

[addendum]

If the editor is not already open, you can press the "New Script" button from the Home tab to open the editor and then follow the instructions above.

To determine the current file and line number, run dbstack() from the command window.

[Blenndrman]

So do I just keep clicking the 'step' button until it gets to fmincon() and the outputs are displayed in the Command Window?

Thanks for the answer, by the way.

[Adam Danz]

That's one way you can do it.

Another way is to open fmincon and put a break point at the line after your desired outputs. Then press "continue" instead of step and the code will execute until it reaches the break point. Then it will pause at the break point.

All variables in the current workspace are available to save. So one option is to simply save all of the variables once you get to the desired area of your code. Another way is to specify which variables need saved. You may want to do both.

Just be careful and make sure you don't press the "Quit Debugging" button, or shift+F5, because then you'll quit execution and lose everything. Hopefully your long-running code is iteratively backing up the data anyway (as should be done with all long-runing code).

[Blenndrman]

Good to know about various tricks to store variables along the way. I'm certainly not a programmer by trade, and have never built such an unwieldy code before. Thanks!

[Adam Danz]

For these kinds of long running processes it's a good idea to occasionally save the data from within the program. For example, every 'n' iterations save all variables so that if the program breaks 10 days from now you can start off at the most recently saved state.

Another good idea is to set the "pause on errors" option (from the Editor in r2019a, click the down arror under 'Run" and select Pause on errors). That way if there is an error the code will pause at the error before breaking. All of the internal variables will hold their values which gives you a chance to do a postmortem in order to debug and fix the problem (and again, save the variables).

A guy in my lab had to run an analysis that took 2 years on a cluster of computers just because a reviewer asked for an additional figure. You can be sure that program backed up the progress occasionally.

Answer 2

[Walter Roberson]

When you use fmincon(), you can designate an output function separately from plot functions.

The outputfcn will get the current state for each iteration. You can save copies of the best solution you found so far (or all solutions, if you prefer). Then when you click STOP in the plotfcn, the outputfcn should in theory execute once with 'done' state, which you can then use to save the values to somewhere the user can get to.

[Blenndrman]

So this will be for the next time I run the code, certainly – I'll be able to set this up. I guess I was wondering about a currently-still-running script. Thanks, though, It's a useful tool that I didn't realize existed.