How to save 'display-iter' from fmincon to a .txt file?

In the options of a fmincon function I set Display to iter and this is how the output loos like. I would like to save it to a .txt file in the very same form that is displayed on a screen. I googleded it and Someone mentioned that the similar problems might be done by using 'OutputFcn' but I have no idea how to do it as I am not sure how my output function should look like.

Thank you in advance!

No active inequalities. Elapsed time is 0.203401 seconds.

                                Max     Line search  Directional  First-order 
 Iter F-count        f(x)   constraint   steplength   derivative   optimality Procedure 
    0      5      2496.66            0                                         
    1     13      2496.25            0        0.125        -14.5         26.9   
    2     20      2495.99      -0.1969         0.25        -2.45         21.8   
    3     26      2495.24      -0.2492          0.5        -2.74         29.9   
    4     33      2494.38      -0.3498         0.25        -7.68         11.1   
    5     38      2494.17      -0.2437            1        -11.2         10.8   
    6     43      2493.36       -0.283            1        -4.19         3.59   
    7     48      2492.84       -0.162            1        -1.31         9.38   
    8     53      2492.63            0            1        -2.28          4.2   
    9     58      2492.61            0            1        -1.28         0.21   
   10     63      2492.61            0            1      -0.0412        0.111   
   11     68      2492.57            0            1      -0.0386         2.94  Hessian modified  
   12     73      2492.52            0            1      -0.0279         5.33   
   13     78      2492.35            0            1      -0.0271         10.2   
   14     83         2492            0            1       -0.025         15.5   
   15     88      2491.35            0            1      -0.0244         18.5   
   16     93      2490.79            0            1      -0.0234         11.4   
   17     98      2490.64            0            1      -0.0642          3.4   
   18    103      2490.61            0            1     -0.00727        0.151   
   19    108      2490.61            0            1     -0.00048      0.00781

Answer 1

[Les Beckham]

Getting back to the original question about capturing the output to a text file, I would simply use 'diary' instead of an OutputFcn. But that's just me; I like to keep it simple when I can. :)

[Stephen23]

LuC's "Answer" moved here:

Thank you, using diary seems to be the easiest way. :)

Answer 2

[Brendan Hamm]

Without having tested this the basic idea you would define the function in the manner described here. You can see the expected syntax and the fields stored in the optimValues input to this function. I give an example of a few inputs:

function stop = outputFcn(x,optimValues,state,fileID)
    fprintf(fileID,'%u,%u,%4.2f \r\n',optimValues.iteration,optimValues.funccount, optimValues.fval)
end

If you are not familiar with fprintf, please see the doc .

Also notice we have an extra input fileID to our output function, so we need to make an anonymous function which we will pass fileID to.

fileID = fopen('myLog.txt','a'); % 'a' will append to a file or create it if it doesn't exist.

f = @(x,optimValues,state) outputFcn(x,optimValues,state,fileID);

fileID is now embedded in the anonymous function's workspace (the workspace of f). Pass it to optimoptions and then pass that on to fmincon.

opt = optimoptions('fmincon','OutputFcn',f);
fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,opt); % I have not defined the parameters here.

[LuC]

Thanks for the prompt reply!

It this written in a correct order?

    if verLessThan ('matlab','7.8')
        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','none') ;
    else
        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'Algorithm','active-set',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','iter',...
            'OutputFcn', f );                        
            %'PlotFcns', {@optimplotx, @optimplotfval, @optimplotfunccount}) ;
    .
    .
    .

        % Solve the problem
        tic
        [Param,fval,exitflag,output] = fmincon (@Cost_Fcn,...
            Param0,[],[],[],[],...
            LB,UB,...
            @Boundary_Fcn,...
            options,...
            Coeff,...
            ListCoeffVariable,...
            FitStrategy.CostRange,...
            Xref,Yref,Timeref,Selected) ;
        vrijeme = toc;

        fileID = fopen( 'myLog.txt', 'a' ); 
        f = @(x,optimValues,state,fileID) outputFcn(x,optimValues,state,fileID);

and outside of this function I defined

function stop = outputFcn(x,optimValues,state, fileID)
      fprintf('Iteration: \t, f(x): \t, Directional derivative: \t, FirstOrderOpt: \n');
      fprintf(fileID,'%u, %u, %4.2f, %4.2f \r\n',optimValues.iteration,optimValues.fval, optimValues.directionalderivative, optimValues.firstorderopt);
      fclosef(fileID);

I got the following error: @(x,optimValues,state,fileID)outputFcn(x,optimValues,state,fileID) Too many input arguments.

Honestly I can't see why there are too many input arguments.

[Brendan Hamm]

In the line where you define f you should have:

f = @(x,optimValues,state) outputFcn(x,optimValues,state,fileID);

Not including the fileId as an input to the function handle f is what makes it an anonymous function. That is f takes the 3 inputs which fmincon expects it to, but it will have the fileID embeded into the workspace of f.

Also, you should define fileId and f before the call to optimoptions which uses f:

if verLessThan ('matlab','7.8')
        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','none') ;
    else
        fileID = fopen( 'myLog.txt', 'a' ); 
        f = @(x,optimValues,state) outputFcn(x,optimValues,state,fileID);
        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'Algorithm','active-set',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','iter',...
            'OutputFcn', f );

Also do not call the line:

fclosef(fileID);

inside of outputFcn as this function is called each iteration of the optimization routine and on the second iteration you would need the file to be open for writing. Instead call fclose after the optimization runs.

[LuC]

Hello Brendan,

I modified the code in a way you told me to do it but I am still getting the same error. I have also tried modifying the code so that it opens fileID everytime I call the function, i.e.

   function stop = myfun(x,optimValues,state)
       fid = fopen( 'myLog.txt', 'a' );
       fprintf('Iteration: \t, f(x): \t, Directional derivative: \t, FirstOrderOpt: \n');
       fprintf(fid,'%u, %u, %4.2f, %4.2f \r\n',optimValues.iteration,optimValues.fval, optimValues.directionalderivative, optimValues.firstorderopt);
  %     fclose(fid);

but I get the same error

Error using PCK/Fit_Sin_Function>@(x,optimValues,state)myfun(x,optimValues,state) Too many input arguments.

[Brendan Hamm]

If you can please post the modified code you are using to run this and if possible the data you are using (or some made up data).

[LuC]

        fid = fopen('myLog.txt','a+');
        f = @(x,optimValues,state) myfun(x,optimValues,state,fid);
        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'Algorithm','active-set',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','iter',...
            'OutputFcn', f); 
- - - - - -
        tic
        [Param,fval,exitflag,output] = fmincon (@Cost_Fcn,...
            Param0,[],[],[],[],...
            LB,UB,...
            @Boundary_Fcn,...
            options,...
            Coeff,...
            ListCoeffVariable,...
            FitStrategy.CostRange,...
            Xref,Yref,Timeref,Selected) ;
        time = toc;
- - - - - - -
        myfun(x,optimValues,state,fid); 
        fclose(fid);
- - - - - - -
 function stop = myfun(x,optimValues,state,fid)
     %fid = fopen( 'myLog.txt', 'a' );
     fprintf('Iteration: \t, f(x): \t, Directional derivative: \t, FirstOrderOpt: \n');
     fprintf(fid,'%u, %u, %4.2f, %4.2f \r\n',optimValues.iteration,optimValues.fval, optimValues.directionalderivative, optimValues.firstorderopt);
     %fclose(fid);

       x        y
   -2.0090  -35.0710
   -2.0080  -31.2590
   -2.0080  -30.8820
   -2.0070  -30.4810
   -2.0010  -21.4850
   -1.9730   -7.1020
   -1.9000   -0.6190
   -1.7710   -8.8840
   -1.6010  -26.5150
   -1.4240  -39.9870
   -1.2590  -38.5350
   -1.1050  -29.2740
   -0.9500  -28.0380
   -0.7870  -35.6560
   -0.6220  -41.3930
   -0.4590  -42.4380
   -0.3000  -42.0180
   -0.1420  -41.3620
    0.0190  -42.8060
    0.1810  -46.8950
    0.3420  -48.0330
    0.5030  -44.2170
    0.6630  -41.3610
    0.8230  -40.8810
    0.9840  -38.9310
    1.1460  -36.4660
    1.3060  -35.6040
    1.4670  -33.6470
    1.6280  -30.5850
    1.7890  -29.7060
    1.9490  -28.8190
    2.1100  -25.1340
    2.2710  -22.8480
    2.4330  -23.7890
    2.5940  -23.9550
    2.7550  -23.5470
    2.9150  -24.8020
    3.0760  -26.0030
    3.2370  -26.6650
    3.3980  -28.1070
    3.5590  -26.3200
    3.7200  -19.9730
    3.8800  -17.1940
    4.0410  -21.8930
    4.2020  -26.9210
    4.3620  -28.9170
    4.5230  -31.6800
    4.6840  -35.0160
    4.8450  -37.0100
    5.0060  -39.2880

Answer 3

[Brendan Hamm]

Ok I see only 2 things wrong here, but there may be more as this obviously isn't ALL of the code.

1. I missed the fact that you need to set the output of your OutputFcn. Inside myfun just set

stop = false;

2. What are all of these extra inputs you are trying to pass to fmincon?

[Param,fval,exitflag,output] = fmincon (@Cost_Fcn,...
            Param0,[],[],[],[],...
            LB,UB,...
            @Boundary_Fcn,...
            options,... %%%%%<- This is the last input to the function
            Coeff,...   %%%%%<- This variable will trigger too many inputs.
            ListCoeffVariable,...
            FitStrategy.CostRange,...
            Xref,Yref,Timeref,Selected) ;

You can't just keep passing more inputs to this function, it only accepts a finite number of inputs in a pre-specified order. This is likely your main cause for error.

3. I can't actually see it, but I imagine that your entire file is a function itself? If not you cannot define a function at the end of it. But this would be the source of your error.

4. This is not going to cause errors, but the way your outputFcn is written it will print your header line at every iteration. I would write that one line directly after opening the file, but not in the outputFcn. Worry about this after your problem actually runs though.

[LuC]

1. Why it has to be set to false, i.e. can it be set to true as well? Or to some random integer?

2./3. When I pass all these extra parameters to fmnincon, everything works perfectly fine if I don't include this line: ('OutputFcn', f).

In my file there is one big function and then the rest of the functions I am calling in this big one, i.e.

function [MacroCoeff,Model,isFliped,Comment] = Fit_Sin_Function (Model,Type,FitStrategy,DISPLAY_FIGURE,MODELNAME,PCK_COEFF) is the 'main one'

and then the stop one is defined in the same file but below the big one. The file itself has about 1100 lines of code and there are no errors when the options are set as following:

        options = optimset ('fmincon') ;
        options = optimset (options,...
            'LargeScale','off',...
            'Algorithm','active-set',...
            'MaxFunEvals',2000,...
            'MaxIter',2000,...
            'Display','iter')
            %'OutputFcn', f);                      
            %'PlotFcns', {@optimplotx, @optimplotfval, @optimplotfunccount}) ;

4. Yes, you are right, it makes no sense to write in every time.

Thank you for the help, I appreciate it!

[Brendan Hamm]

1. If set to true this will terminate the underlying optimization on that iteration.

2. I think I realize what is going on here. These extra values are parameters to your objective function and boundary. Is this correct? If so we simply need to add an extra input to your OutputFcn for each value you have here.

Since you have 7 extra inputs to the fmincon function: Coeff,ListCoeffVariable,FitStrategy.CostRange,Xref,Yref,Timeref,Selected

These are passed to your objective function, your non-linear constraint function, and any other function fmincon calls including the OutputFcn. This is not the preferred way to do this (I'll get back to this), but I can show you how to solve this now:

Create 7 inputs for the values which fmincon will pass and then fID last

    function stop = myfun(x,optimValues,state,~,~,~,~,~,~,~,fID)
         stop = false;

         fprintf(fid,'%u, %u, %4.2f, %4.2f \r\n',...
               optimValues.iteration,...
               optimValues.fval,...
               optimValues.directionalderivative,...
               optimValues.firstorderopt);
    end

f = @(x,optimValues,state,in1,in2,in3,in4,in5,in6,in7) ...
myfun(x,optimValues,state,in1,in2,in3,in4,in5,in6,in7,fid);

Also note that directionalderivative is not going to be in optimValues for the fmincon solver. So remove this from this function.

P.S. The preferred way of passing extra inputs to the objective and non-linear constraint functions is with an anonymous function handle. Doing this would have made this whole issue irrelevant and is why we no longer document the extra input arguments in fmincon . Please see the documentation regarding this.

I will post a way to use anonymous functions everywhere with a simplified example following this.

Note: Changed the anonymous function handles call to myfun from outputFcn. This should be calling the function I define right above.

[Brendan Hamm]

Example to show anonymous function handles:

function out = runMin

% Data generation
rng('default');
% generate some data of form y = 2 -2*x + 0.2*x^2 + epsilon
x = 10*rand(1000,1);
X = [ones(size(x)) x x.^2]; 
trueCoeff = [2;-2;0.2];
y = X*trueCoeff + randn(size(x)); % Sample y values

% Note I need other variables in my optimization so I embed them in my
% objective function with an anonymous function. obj now takes 1 input as
% expected by fmincon, but has the variables X and y in its workspace.
obj = @(coeff) objFun(coeff,X,y); 

% Create Log file
fID = fopen('log.txt','a+');
fprintf(fID,'Iteration: \t, f(x): \t, FirstOrderOpt: \n');

% Anonymous function handle to my outputFcn
outF = @(x,optimValues,state) outFun(x,optimValues,state,fID);

opt = optimset('fmincon');
opt = optimset(opt,'outputFcn',outF);
% Better alternative post 2013a release:
% opt = optimoptions('fmincon','outputFcn',outF);

out = fmincon(obj,ones(3,1),[],[],[],[],[],[],[],opt);
fclose(fID);

end

function val = objFun(coeff,X,y)
% Minimize squared difference
yHat = X*coeff; % Fitted y values
val = sum((yHat-y).^2); % Sum squared error
end

function stop = outFun(~,optimValues,~,fID)
% Print out values at every iteration:
fprintf(fID,'%u, %u, %4.2f \r\n',optimValues.iteration,optimValues.fval, optimValues.firstorderopt);
stop = false; % Do not stop the algorithm
end

[LuC]

I have some more errors and this is what I have tried so far.

So first I got this error

    Error: File: Fit_Sin_Function.m Line: 40 Column: 24
    Unexpected MATLAB operator.

and then I tried changing it to:

in1 = FitStrategy.CostRange;

I run the programme and I get this error for the function:

myfun(x,optimValues,state,~,~,~,~,~,~,~,fid);

    Error: File: Fit_Sin_Function.m Line: 381 Column: 36
    Expression or statement is incorrect--possibly unbalanced (, {, or [.

I tried changing this to

myfun(x,optimValues,state,Coeff,ListCoeffVariable, ...
            in1, Xref, Yref,...
            Timeref, Selected,fid);

and now again I get a new error

Reference to non-existent field 'CostRange'.

Error in PCK/Fit_Sin_Function (line 27)
    in1 = FitStrategy.CostRange;

The big function where this is all happening accepts FitStrategy as an argument from the main programme.

function [MacroCoeff,Model,isFliped,Comment] = Fit_Sin_Function (Model,Type,FitStrategy,DISPLAY_FIGURE,MODELNAME,PCK_COEFF)

There are so many complications with the OutputFcn, I guess it would be much easier to do c/p manually into a .txt file.

[Brendan Hamm]

It is absolutely impossible to debug the source of errors without knowing what is on Line: 40 Column: 24 where you use an unexpected operator.

This is a similar issue with all of your other errors and the changes you mention. Are you changing the definition of myFun or the assignment to the anonymous function handle? Is the error about unbalanced parenthesis on the line you show? Where are you defining the assignment in1 = FitStategy.CostRange, inside the function declaration or in the Fit_Sin_Function?

The issue is not with the outputFcn, you can see my example above works exactly as expected. Maybe you should consider using the coding approach I mention rather than the extra input arguments to fmincon.

[LuC]

The unexpected operator error is in FitStrategy.CostRange and thus I changed it to in = FitStrategy.CostRange.

Okay, I will try that approach. Thank you