intlinprog
primal-simplex
algorithm The 'primal-simplex'
algorithm of the intlinprog
solver can solve more problems
than before, and has better performance on large problems. Select
the algorithm in the RootLPAlgorithm
option.
intlinprog
iterations can differ from previous
versions. This holds even when you select the 'dual-simplex'
algorithm,
because intlinprog
uses the 'primal-simplex'
algorithm
for some calculations in any case.
linprog dual-simplex
algorithm returns more
informationThe linprog
'dual-simplex'
algorithm
now returns a Lagrange multiplier structure and an output
structure
containing the first-order optimality measure.
quadprog active-set
algorithm will be removed The quadprog
'active-set'
algorithm
now warns that it will be removed in a future release.
To avoid this warning, use optimoptions
to
set the Algorithm
option to 'interior-point-convex'
or 'trust-region-reflective'
.
Or simply do not set the Algorithm
option; quadprog
defaults
to the 'interior-point-convex'
algorithm.
fmincon
allows problems without constraints fmincon
no longer throws
an error if you run it on a problem without constraints. This change
can make it easier for you to see the effect of constraints by running
a problem both with and without constraints.
To run fmincon
without constraints, you
no longer need to set an artificial constraint, such as lb
= -Inf
, but doing so will not matter.
Any code that checks for this error will have different behavior than before.
The lsqnonlin
and lsqcurvefit
solvers now allow you to fix
variables by specifying equal upper and lower bounds. For example,
lower bound lb = [0,0]
and upper bound ub
= [Inf,0]
fix the solution x
to have x(2)
= 0
.
quadprog
allows zero or empty quadratic
termPreviously, if you gave a zero or empty matrix as the H
input,
the quadprog
solver would switch
to linprog
. Now quadprog
solves
the problem.
The algorithm
field of the output
structure
of several solvers has changed. Now, all output.algorithm
strings
are the same as those you set in the Algorithm
name-value
pair. For example, the output.algorithm
field of
the fmincon
'active-set'
algorithm
used to be 'medium-scale: SQP, Quasi-Newton, line-search'
.
Now it is 'active-set'
.
The affected solvers are fgoalattain
, fminimax
, fmincon
, fseminf
, fsolve
, linprog
, lsqcurvefit
,
and lsqnonlin
.
Any code that uses the string in output.algorithm
can
have different behavior than before.
There is a new featured example of mixed-integer quadratic programming, Mixed-Integer Quadratic Programming Portfolio Optimization.
linprog
linear
programming solverlinprog
has
a new algorithm option named 'dual-simplex'
. The
algorithm saves memory when you specify sparse constraint matrices.
For details, see the function reference page.
Currently, 'dual-simplex'
returns an empty
Lagrange multiplier structure lambda
and firstorderopt
field
in the output
structure.
lsqlin
linear
least-squares solverlsqlin
has
a new algorithm option named 'interior-point'
.
The algorithm handles all types of constraints and saves memory when
you specify sparse constraint matrices. For details, see the function
reference page.
intlinprog
solverintlinprog
accepts OutputFcn
and PlotFcns
options.
For details, see intlinprog
Output Functions and Plot Functions.
There is a built-in output function that collects all integer
feasible solutions that intlinprog
encounters.
Choose this output function by using optimoptions
to
set the OutputFcn
option to @savemilpsolutions
.
There is a built-in plot function that shows details of the
solver iterations. Choose this plot function by using optimoptions
to
set the PlotFcns
option to @optimplotmilp
.
InitDamping
option The fsolve
, lsqcurvefit
,
and lsqnonlin
solvers
can use the 'levenberg-marquardt'
algorithm. To
initialize the Levenberg-Marquardt parameter differently than its
default, set the new InitDamping
option using optimoptions
.
Previously, to initialize the Levenberg-Marquardt parameter
you would pass a cell array in the Algorithm
option
when using optimset
,
such as
options = optimset('Algorithm',{'levenberg-marquardt',0.1});
That method still works. But when using optimoptions
,
you cannot set the parameter in the Algorithm
option,
and instead must use the InitDamping
option, such
as
options = optimoptions(@fsolve,'Algorithm','levenberg-marquardt','InitDamping',0.1);
fminunc
The fminunc
default
algorithm will change to 'quasi-newton'
in a future
release.
fminunc
now warns when you run it in cases
where the default behavior will change. For details, see the fminunc
function
reference page or fminunc
Algorithms.
There is a new featured example of mixed-integer linear programming, Optimal Dispatch of Power Generators.
linprog
algorithms will be removed in
the future The linprog
'active-set'
and 'simplex'
algorithms
warn that they will be removed in a future release.
To avoid this warning, choose the 'interior-point'
or 'dual-simplex'
algorithms
using optimoptions
.
fminunc
options will be removed in the
future fminunc
now
warns that the InitialHessMatrix
and InitialHessType
options
will be removed in a future release.
To avoid these warnings, do not set values for these options. The Optimization app no longer has these options.
bintprog
removed The bintprog
function has been removed.
To update code to use intlinprog
instead
of bintprog
, see Tips.
ktrlink
removed The ktrlink
interface to the KNITRO^{®} third-party
solver has been removed.
For a KNITRO interface, contact Ziena Optimization: www.ziena.com
.
The intlinprog
function
solves mixed-integer linear programming problems. For more information,
see the reference page and the documentation.
There are several new featured examples of mixed-integer linear programming:
Currently, you cannot run intlinprog
in
the Optimization
app.
fmincon
and quadprog
When you do not specify an algorithm, fmincon
and quadprog
default
to different algorithms than before. Usually, the new algorithms are
faster and more robust than the algorithms they replace.
Solver | Previous Default Algorithm | New Default Algorithm |
---|---|---|
fmincon | 'trust-region-reflective' | 'interior-point' |
quadprog | 'trust-region-reflective' | 'interior-point-convex' |
Solvers can produce different results than before. To reproduce
previous results, set the Algorithm
option with optimoptions
.
options = optimoptions('fmincon','Algorithm','trust-region-reflective'); % or options = optimoptions('quadprog','Algorithm','trust-region-reflective');
Be sure to pass options
in your function
call.
x = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nlcon,options) % or x = quadprog(H,f,A,b,Aeq,beq,lb,ub,x0,options)
The lsqcurvefit
and lsqnonlin
solvers
have slightly different behavior than before when the internally calculated
trust-region radius gets large in the 'trust-region-reflective'
algorithm,
or the Levenberg-Marquardt parameter gets small in the 'levenberg-marquardt'
algorithm.
There are no longer any arbitrary limits on these parameters. fsolve
has
the same change to its 'trust-region-reflective'
and 'levenberg-marquardt'
algorithms.
The UseParallel
option now accepts the values true
and false
.
The option also accepts the former values 'always'
and 'never'
,
and scalar values 1
and 0
.
The affected solvers are fmincon
, fgoalattain
,
and fminimax
.
ktrlink
default math library change The ktrlink
function
uses a different default math library within KNITRO than before.
This change can enable ktrlink
to run on more
hardware versions.
With the new default math library, ktrlink
runs
slower than before. To use the previous math library, set the KNITRO format
option blasoption
to 1. See Setting
Options.
bintprog
will be removed in the futureintlinprog
solves
more problems than bintprog
,
and has better performance. So bintprog
will
be removed in a future release.
To update your existing bintprog
code to
use intlinprog
, see Tips.
ktrlink
will be removed in the futurektrlink
will
be removed in a future release. For an updated KNITRO interface,
contact Ziena Optimization: www.ziena.com
.
All nonlinear solvers now check whether derivatives of the objective
and nonlinear constraint functions are well defined at the initial
point, usually called x0
. (This is in addition
to the existing checks that the functions are well defined at x0
.)
Well defined means the value of each derivative is not NaN
, Inf
,
or complex (nonlinear least-squares solvers allow complex values).
Derivatives include both gradients and Jacobians. See Including
Derivatives and Writing
Vector and Matrix Objective Functions.
When the objective or nonlinear constraint functions do not
include a derivative, solvers approximate derivatives by finite differences.
This means that the functions must be well defined for points in a
small neighborhood of x0
.
If any derivative is not well defined at x0
,
the solver stops with an error, and does not attempt to find a solution.
In all tested cases, this behavior led to a clearer exit condition.
It is conceivable that a problem that previously ran to completion will now exit without completion. This can occur in the rare case when a derivative did not exist for a function at the initial point, but the solver was able to step to a point where the derivatives were well defined.
optimoptions
function for setting options
with compact and comprehensive display The new optimoptions
function
creates and modifies options for all solvers except the base MATLAB^{®} solvers fminbnd
, fminsearch
, fzero
,
and lsqnonneg
. Continue to use optimset
for
those functions.
optimoptions
organizes options by solver,
with a more focused and comprehensive display than optimset
:
Creates and modifies only the options that apply to a solver
Shows your option choices and default values for a specific solver/algorithm
Displays links for more information on solver options and other available solver algorithms
For details, see the optimoptions
page,
or Set Options.
If you export options or a problem from the Optimization Tool,
the options are an object as created by optimoptions
.
Therefore, earlier software versions cannot import the options or
problem. This consideration does not apply to the base MATLAB solvers,
which continue to encapsulate options as a structure.
Algorithm
option replaces LargeScale
option Use the Algorithm
option to select the algorithm
in all solvers that have multiple algorithms. In a future release,
solvers will no longer use the LargeScale
option.
The linprog
Simplex
option
will also change to the Algorithm
option.
Update options as follows.
Solver | Previous Setting | Current Setting |
---|---|---|
fminunc | 'LargeScale' = 'on' | 'Algorithm' = 'trust-region' |
'LargeScale' = 'off' | 'Algorithm' = 'quasi-newton' | |
linprog | 'LargeScale' = 'on' | 'Algorithm' = 'interior-point' |
'LargeScale' = 'off' , 'Simplex'
= 'off' or unset | 'Algorithm' = 'active-set' | |
'LargeScale' = 'off' , 'Simplex'
= 'on' | 'Algorithm' = 'simplex' | |
lsqlin | 'LargeScale' = 'on' | 'Algorithm' = 'trust-region-reflective' |
'LargeScale' = 'off' | 'Algorithm' = 'active-set' |
ktrlink
supports KNITRO 8.1The ktrlink
function now supports KNITRO version
8.1. For details, see ktrlink:
An Interface to KNITRO Libraries.
fmincon
and quadprog
The default fmincon
algorithm
will become 'interior-point'
.
The default quadprog
algorithm
will become 'interior-point-convex'
.
These solvers now warn when you run them in cases where the default behavior will change. For example, they warn when:
You do not set the Algorithm
option.
You set incompatible LargeScale
and Algorithm
options.
To avoid these warnings:
Do not set the LargeScale
option.
Set the Algorithm
option appropriately.
For details, see the fmincon
and quadprog
function
reference pages or Choosing
the Algorithm.
ktrlink
supports KNITRO 8The ktrlink
function
now supports KNITRO version 8. For details, see ktrlink:
An Interface to KNITRO Libraries.
The fminunc
medium-scale
algorithm now attempts to recover from failures when evaluating the
objective function during iteration steps, or during gradient estimation.
Failure means the objective function returns NaN
,
a complex value, or Inf
. If there is such a failure,
the algorithm attempts to take different steps.
As part of robustness, the fminunc
medium-scale
algorithm now uses the ObjectiveLimit
tolerance.
When objective function values drop below ObjectiveLimit
(default
value: -1e20
), iterations end with a -3
exit
flag. Use optimset
to
change the value of ObjectiveLimit
. Set ObjectiveLimit
to -Inf
to
disable this tolerance.
The FinDiffRelStep
option for choosing relative
finite difference step sizes is now available in the Optimization
Tool, in the Approximated derivatives pane. This
option lets you tune the gradient estimation step in most solvers.
The fsolve
, lsqcurvefit
,
and lsqnonlin
solvers
no longer use the magnitude of the Levenberg-Marquardt regularization
parameter as a stopping criterion, so they no longer return an exit
flag of -3
when using the levenberg-marquardt
algorithm.
Instead, they use the TolX
tolerance in all internal
calculations.
The solvers now stop with exit flag 2
in
most situations where previously they stopped with exit flag -3
.
The fmincon
sqp
algorithm
calculates its Lagrange multiplier estimates somewhat differently
than before.
The fmincon
sqp
algorithm
can give slightly different results than before.
The fsolve
, lsqcurvefit
,
and lsqnonlin
solvers
now accept the FinDiffType
option. Set FinDiffType
to 'central'
with optimset
to
enable derivative estimation by central finite differences. Central
finite differences are more accurate, but take more time than the
default 'forward'
finite differences.
fsolve
, lsqcurvefit
,
and lsqnonlin
now
use the TypicalX
option when estimating dense Jacobians
via finite differences. In previous releases, these solvers used TypicalX
only
when checking derivatives.
For algorithms that obey bounds, finite difference steps for derivative estimation now stay within any bounds you set for the decision variables. See Iterations Can Violate Constraints.
The new FinDiffRelStep
option allows
you to set a vector of finite difference step sizes to better handle
problems whose components have different scales. Use FinDiffRelStep
at
the command line for any solver that uses finite differences. For
details, see FinDiffRelStep
in Options
Structure.
The fsolve
, lsqcurvefit
,
and lsqnonlin
functions
no longer use the Gauss-Newton algorithm.
The previous way of selecting the Gauss-Newton algorithm was
to set the LargeScale
option to 'off'
,
and in:
fsolve
— set the NonlEqnAlgorithm
option
to 'gn'
.
lsqcurvefit
or lsqnonlin
—
set the LevenbergMarquardt
option to 'off'
.
To select an algorithm, use optimset
to
set the Algorithm
option:
fsolve
— trust-region-dogleg
, trust-region-reflective
,
or levenberg-marquardt
lsqcurvefit
or lsqnonlin
— trust-region-reflective
or levenberg-marquardt
Solvers no longer use the LevenbergMarquardt
, LineSearchType
,
and NonlEqnAlgorithm
options, since these options
relate only to the Gauss-Newton algorithm.
The DerivativeCheck
option checks whether
a solver's finite-difference approximations match the gradient
or Jacobian functions that you supply. When a solver finds a discrepancy
between the computed derivatives and their finite-difference approximations,
the solver now errors. Solvers used to pause in this situation instead
of erroring.
Additionally, solvers now compare derivatives at a point near
the initial point x0
, but not exactly at x0
.
Previously, solvers performed the comparison at x0
.
This change usually gives more reliable DerivativeCheck
decisions.
For details, see Checking
Validity of Gradients or Jacobians.
Solvers do not include the computations for DerivativeCheck
in
the function count. See Iterations
and Function Counts.
Solvers compare the derivatives at a different point than before, so can change their decision on whether the derivatives match. Solvers now error instead of pause when they encounter a discrepancy.
The fmincon
interior-point
and sqp
algorithms
can use the ScaleProblem
option. The default value
of ScaleProblem
is now 'none'
instead
of 'obj-and-constr'
.
Because of a bug in previous releases, when you did not provide
gradients of the objective and nonlinear constraint functions, fmincon
did
not scale these functions. fmincon
did scale
linear constraints. So, if you do not provide gradients and have no
linear constraints, the current fmincon
behavior
is the same as in previous releases. However, the current behavior
can differ if you do provide gradients (GradObj
or GradConstr
is 'on'
).
If you provide gradients, have no linear constraints, and want to
obtain the previous behavior, set ScaleProblem
to 'obj-and-constr'
with optimset
.
The fsolve
trust-region-dogleg
algorithm
no longer performs an internal calculation of conditioning. This change
usually speeds fsolve
.
fsolve
iterations
differ from previous versions. Additionally, the solution and all
associated outputs can differ from previous versions. Usually, results
are numerically equivalent to previous results.
For R2011b, error and warning message identifiers have changed in Optimization Toolbox™.
If you have scripts or functions that use message identifiers
that changed, you must update the code to use the new identifiers.
Typically, message identifiers are used to turn off specific warning
messages, or in code that uses a try
/catch
statement
and performs an action based on a specific error identifier.
For example, the 'optim:fmincon:ConstrainedProblemsOnly'
identifier
has changed to 'optimlib:fmincon:ConstrainedProblemsOnly'
.
If your code checks for 'optim:fmincon:ConstrainedProblemsOnly'
,
you must update it to check for 'optimlib:fmincon:ConstrainedProblemsOnly'
instead.
To determine the identifier for a warning, run the following command just after you see the warning:
[MSG,MSGID] = lastwarn;
This command saves the message identifier to the variable MSGID
.
To determine the identifier for an error, run the following command just after you see the error:
exception = MException.last; MSGID = exception.identifier;
Tip Warning messages indicate a potential issue with your code. While you can turn off a warning, a suggested alternative is to change your code so it runs warning free. |
quadprog
has
a new algorithm named 'interior-point-convex'
.
It has these features:
The algorithm has fast internal linear algebra.
The algorithm handles sparse problems.
There is a new presolve module that can improve speed, numerical stability, and detection of infeasibility.
The algorithm handles large convex problems, and accepts and uses sparse inputs. See Large-Scale vs. Medium-Scale Algorithms.
The algorithm optionally gives iterative display.
The algorithm has enhanced exit messages.
For details on the algorithm, see interior-point-convex quadprog Algorithm. For help choosing the algorithm to use, see Quadratic Programming Algorithms.
You now choose the quadprog
algorithm by
using optimset
to
set the Algorithm
option instead of the LargeScale
option.
If you don't set Algorithm
or LargeScale
, quadprog
behaves
as before.
Algorithm
option choices are:
trust-region-reflective
(formerly LargeScale
= 'on'
), the default
active-set
(formerly LargeScale
= 'off'
)
interior-point-convex
The previous way of choosing the quadprog
algorithm
at the command line was to set the LargeScale
option
to 'on'
or 'off'
. quadprog
now
ignores the LargeScale
option, except when you
set the inconsistent values LargeScale
= 'off'
and Algorithm
= 'trust-region-reflective'
.
In this case, to avoid backward incompatibility, quadprog
honors
the LargeScale
option, and uses the 'active-set'
algorithm.
quadprog
now checks whether any inputs
are complex, and, if so, it errors. The only exception is the Hinfo
argument
for the HessMult
option is allowed to be complex.
More solvers now attempt to recover from errors in the evaluation
of objective functions and nonlinear constraint functions during iteration
steps, or, for some algorithms, during gradient estimation. The errors
include results that are NaN
or Inf
for
all solvers, or complex for fmincon
and fminunc
.
If there is such an error, the algorithms attempt to take different
steps. The following solvers are enhanced:
fmincon
trust-region-reflective
algorithm
(the interior-point
and sqp
algorithms
already had this robustness)
fminunc
LargeScale
algorithm
fsolve
trust-region-reflective
, trust-region-dogleg
,
and levenberg-marquardt
algorithms
lsqcurvefit
trust-region-reflective
and levenberg-marquardt
algorithms
lsqnonlin
trust-region-reflective
and levenberg-marquardt
algorithms
The DiffMinChange
and DiffMaxChange
options
set the minimum and maximum possible step sizes for finite differences
in gradient estimation. The defaults are now:
DiffMinChange = 0
(formerly 1e-8
)
DiffMaxChange = Inf
(formerly 0.1
)
Solvers have mechanisms that ensure nonzero and non-infinite step sizes, so the new defaults simply mean that the step size adjustment algorithms have fewer constraints.
The new defaults remove the previous arbitrary choices. The previous values can be inappropriate when components are too large or small in magnitude. Tests show these new defaults are good for most situations.
Some solver iterations can differ from previous ones. To obtain the previous behavior:
options = optimset('DiffMinChange',1e-8,'DiffMaxChange',0.1);
For the trust-region-reflective
algorithm,
the algorithm
field of the output
structure
is now 'trust-region-reflective'
. This value differs
slightly from the previous values returned by fmincon
, fsolve
, lsqcurvefit
, lsqnonlin
,
and quadprog
.
To avoid errors or unexpected results, update any code that
depends on the exact value of the output.algorithm
string.
ktrlink
is
compatible with KNITRO 7.
For details, see ktrlink: An
Interface to KNITRO Libraries, or the Ziena Optimization web
site http://www.ziena.com/
.
The fmincon
interior-point
and sqp
algorithms
now attempt to recover from errors in the evaluation of objective
functions and nonlinear constraint functions during gradient estimation.
The errors include results that are NaN
, Inf
,
or complex. If there is such an error, the finite differencing routines
attempt to take different steps.
The ktrlink
function
now works with Macintosh 64-bit systems. Therefore, ktrlink
works
on the same systems as all other Optimization Toolbox functions.
All linprog
and quadprog
algorithms
now create a firstorderopt
field in the output
structure. This field contains the value of the first-order
optimality measure at the final point.
All fmincon
and quadprog
algorithms
now create a constrviolation
field in the output
structure. This field contains the largest value of the constraint
functions at the final point: bounds, linear constraints, and nonlinear
constraints. (Some algorithms return the larger of the constraint
functions and 0.) See Writing Constraints.
fmincon
has
a new algorithm called SQP
for Sequential Quadratic
Programming. The algorithm has the following features:
Honors bounds at all iterations
Attempts a different step if one leads to an objective
or constraint function returning a NaN
, Inf
,
or complex result
Fast internal linear algebra for solving quadratic programs
Choose the algorithm at the command line by setting the Algorithm
option
to 'sqp'
with optimset
.
For more information about the algorithm, see fmincon
SQP Algorithm in the Optimization Toolbox documentation.
The lsqnonneg
solver
no longer accepts a start point x0
as an optional
input.
The Optimization Tool no longer has an input region for accepting
a start point. If you import or run a problem that contains a start
point x0
, MATLAB issues a warning. Also, the
Optimization Tool and lsqnonneg
ignore x0
,
and instead use a start point of a vector of zeroes. If you export
a problem structure from the Optimization Tool, there is no x0
field.
Enhanced, clearer exit messages in fsolve
, lsqnonlin
,
and lsqcurvefit
, with links for more information.
For more information about the enhancements, see Exit Flags and
Exit Messages.
For solvers with enhanced exit messages, the content of output.message
contains
many more characters than before. User code that relies on this field
might need to be modified in order to display the larger exit message
satisfactorily.
The fmincon
interior-point algorithm attempts
to continue when a user-supplied objective or constraint function
returns Inf
, NaN
, or a complex
result. For more information, see fmincon
Interior Point Algorithm.
The large-scale quadprog
algorithm now
uses the TolFun
and MaxIter
tolerances
for deciding when to end iterations when there are only linear equality
constraints, instead of the TolPCG
and MaxPCGIter
tolerances.
The quadprog
output
structure
now contains the constrviolation
field, which reports
the maximum constraint function at the final point.
For large-scale linear equality constrained problems, the default
values of the tolerances are much tighter than before, so quadprog
can
take more iterations, but the resulting solution should be more accurate.
The large-scale interior-point algorithm of linprog
now
has a backtracking mechanism for the case of stalling, and performs
LDL factorization when there is rank deficiency. For more information,
see Large
Scale Linear Programming.
The linprog
output
structure
now contains the constrviolation
field, which reports
the maximum constraint function at the final point.
The interior-point algorithm of linprog
might
arrive at different solutions than before, and can solve more problems
than before.
The optimValues
structure, used by output
functions, has two new fields to better reflect the state of multiobjective
solvers:
For fgoalattain
,
the optimValues.attainfactor
field contains the
value of γ, the attainment factor.
For fminimax
,
the optimValues.maxfval
field contains the value
max_{i}F_{i},
where F is the vector of objectives.
Furthermore, the value stored in optimValues.fval
has
changed. Now optimValues.fval
contains the vector F of
objective function values. For a complete description of the current optimValues
structure,
see Fields
in optimValues.
User code that uses the optimValues.fval
field
within an output function in fgoalattain
and fminimax
might
need to be updated to avoid errors
The fmincon
solver's interior-point
algorithm
can now compute finite differences in parallel in order to speed the
estimation of gradients. For details on how to use this parallel gradient
estimation, see the Parallel Computing
for Optimization chapter in the User's Guide.
Solvers print exit messages by default at the end of their runs. The exit messages are different in R2009a for several solvers, and the messages have been enhanced with new functionality. The following sections describe the new features and changes. There is more information in the Exit Flags and Exit Messages section of the User's Guide.
The following solvers have enhanced exit messages:
fgoalattain
fmincon
fminimax
fminunc
fseminf
The enhanced exit messages include hyperlinks within their exit messages. These hyperlinks bring up a window containing further information about the terms used in the exit messages.
A <stopping criteria details>
hyperlink
may appear at the end of an exit message, depending on the solver
and setting of the Display
option. This link causes
the solver to print more detail about the exit conditions to the MATLAB Command
Window.
There are new values of the Display
option
to control whether detailed exit messages appear instead of the default
(simpler) messages. The new values are:
'final-detailed'
'iter-detailed'
'notify-detailed'
These settings have the same effect as the corresponding settings
without '-detailed'
, but give detailed exit messages
instead of the default exit messages. For solvers without the new
exit messages, the '-detailed'
options give the
same behavior as without '-detailed'
.
For solvers with enhanced exit messages, the message
field
of the output structure contains both the default (simpler) and the
detailed exit messages, separated by a line of text stating Stopping
criteria details:
. The message field does not contain hyperlinks;
it contains only text.
For solvers with enhanced exit messages, the content of output.message
contains
many more characters than before. User code that relies on this field
may need to be modified in order to display the larger exit message
satisfactorily.
The simplex algorithm of linprog
now detects
when there is no progress in the solution process. It attempts to
continue by performing bound perturbation.
The simplex algorithm of linprog
might
arrive at different solutions than before, and can solve more problems
than before.
One exit flag in the fminunc
medium-scale
solver was changed from –2 to 5. This flag appears when the
solver predicts a change in function value at the next step in its
iterations will be less than the TolFun
tolerance.
This condition can occur at a relative minimum, which should be reported
by a positive flag.
This change might cause users (or code) that examine exit flags to evaluate a result more favorably than previously, since positive exit flags represent normal termination of solvers.
There are two new demos:
A demo showing how to use Symbolic Math Toolbox™ functions
to help calculate gradients and Hessians. Run the demo at the MATLAB command
line by entering echodemo symbolic_optim_demo
.
A demo showing how to use fseminf
for
investigating the effect of parameter uncertainty. Run the demo at
the MATLAB command line by entering echodemo airpollution
.
Furthermore, the optimization tutorial demo now shows how to
include extra parameters. Run the demo at the MATLAB command
line by entering echodemo tutdemo
.
The Levenberg-Marquardt algorithm was refactored in
the solvers fsolve
, lsqcurvefit
and lsqnonlin
.
It is now a more standard implementation, that accepts and preserves
sparse Jacobians.
Choose between the algorithms used in fsolve
, lsqcurvefit
and lsqnonlin
using
the new Algorithm
option.
There is a new ScaleProblem
option
that can sometimes help the Levenberg-Marquardt algorithm converge.
The default fsolve
algorithm, 'trust-region-dogleg'
,
has been validated to work with sparse Jacobians.
The refactored Levenberg-Marquardt algorithm can cause fsolve
, lsqcurvefit
and lsqnonlin
to
yield different answers than before.
The previous way of choosing the algorithm at the
command line was to set the LargeScale
option to 'on'
or 'off'
,
and, for all solvers but fsolve
, to set the LevenbergMarquardt
option
to 'on'
or 'off'
. For fsolve
,
in addition to the LargeScale
option, you needed
to set the NonlEqnAlgorithm
option appropriately. LargeScale
, NonlEqnAlgorithm
,
and LevenbergMarquardt
are now ignored, except
when choosing to use the Gauss-Newton algorithm.
The Gauss-Newton algorithm warns that soon it may no longer be available.
The default value of the MaxFunEvals
option
in the refactored Levenberg-Marquardt algorithm is now 200*numberOfVariables
;
the previous value was 100*numberOfVariables
.
You can now access built-in parallel functionality in Optimization Tool for relevant Optimization Toolbox solvers and, if licensed, Global Optimization Toolbox solvers. The option is available when you have a license for Parallel Computing Toolbox™ functions.
The following solvers can now use central finite differences for gradient estimation:
The fmincon
active-set algorithm and fminunc
medium-scale
algorithm gained central finite differences this release. The fmincon
interior-point
algorithm already had them, and the trust-region-reflective algorithm
for both solvers requires a user-supplied gradient, so does not use
finite differences.
To use central finite differences, use optimset
to
set the FinDiffType
option to 'central'
instead
of the default 'forward'
. This causes the solver
to estimate gradients by formulae such as
$$\nabla f(x)\approx \left[\frac{f(x+{\Delta}_{1}{e}_{1})-f(x-{\Delta}_{1}{e}_{1})}{2{\Delta}_{1}},\dots ,\frac{f(x+{\Delta}_{n}{e}_{n})-f(x-{\Delta}_{n}{e}_{n})}{2{\Delta}_{n}}\right],$$
instead of
$$\nabla f(x)\approx \left[\frac{f(x+{\Delta}_{1}{e}_{1})-f(x)}{{\Delta}_{1}},\frac{f(x+{\Delta}_{2}{e}_{2})-f(x)}{{\Delta}_{2}},\dots ,\frac{f(x+{\Delta}_{n}{e}_{n})-f(x)}{{\Delta}_{n}}\right].$$
Central finite differences take twice as many function evaluations as forward finite differences, but are usually much more accurate.
Central finite differences can work in parallel for gradient
estimation in fgoalattain
, fmincon
active-set
algorithm, and fminimax
.
For details on how to use this parallel gradient estimation, see the Parallel Computing
for Optimization chapter in the User's Guide.
lsqnonneg
was
refactored. It can now use sparse matrices, and it preserves sparsity
during its execution.
A subroutine for gradient estimation by forward finite differences
in nonlinear solvers had a bug that affected it when the current point x
had
a component with the value 0. Forward finite differences are typically
calculated with a step size proportional to sqrt(eps)
,
which is about 1.5*10^{–8}. When a
component of x
was 0, the step size would instead
be proportional to DiffMinChange
, which has a default
value of 10^{–8}. There is now no difference
in step size when x
is 0.
Nonlinear solvers can run slightly differently whenever an iteration
causes a component of x
to be zero, and gradients
are estimated by forward finite differences.
The DerivativeCheck
option enables you to
ascertain whether the derivative (gradient) functions that you supply
for objective or constraint functions give approximately the
same values as those estimated by a solver using finite differences.
The meaning of "approximately" has changed. Now it means
the relative error of each component of the gradient is less than
10^{–6}, unless the size of an analytically
given component is smaller than 1, in which case it means the absolute
difference is less than 10^{–6}. Previously,
the gradients were considered approximately equal if the maximum absolute
error in any component of the gradient was less than (10^{–6} *
norm of analytic gradient) + 10^{–5}.
Some problems will now report violations of the DerivativeCheck
condition,
when previously they would not.
fmincon
, fminimax
,
and fgoalattain
can
take finite differences in parallel in order to speed the estimation
of gradients. For details on how to use this parallel gradient estimation,
see the Parallel
Computing for Optimization chapter in the User's Guide.
The Global Optimization Toolbox GUIs gatool
and psearchtool
have
been combined into the Optimization Tool GUI. To access these GUIs,
type optimtool
at the command line, and choose
the appropriate solver.
Furthermore, three new Global Optimization Toolbox solvers
were added to Optimization Tool: gamultiobj
, simulannealbnd
,
and threshacceptbnd
.
Optimization Tool shows Global Optimization Toolbox solvers only if these solvers are licensed.
The new interior-point algorithm is a large-scale algorithm
that can handle all types of constraints. It has several new options,
explained in the fmincon
function
reference pages.
fmincon
now
has three algorithms. Choose between them by setting the new option Algorithm
to:
'trust-region-reflective'
(formerly
known as 'large scale'
)
'active-set'
(formerly known as 'medium
scale'
)
'interior-point'
By default, Algorithm
= 'trust-region-reflective'
.
The previous way of choosing the algorithm at the command line
was to set option LargeScale
to 'on'
or 'off'
. LargeScale
is
now ignored, except when LargeScale
= 'off'
and Algorithm
= 'trust-region-reflective'
.
In this case, the 'active-set'
algorithm is used,
to minimize backward incompatibility.
Use the new ktrlink
function
to call KNITRO optimization
libraries from Ziena Optimization, Inc. KNITRO libraries must be purchased
separately. The External Interface chapter
of the User's Guide describes the ktrlink
function.
The default value of the PrecondBandWidth
option
changed from 0 to Inf
for the lsqcurvefit
, lsqnonlin
,
and fsolve
solvers.
This change was beneficial in the vast majority of tested problems.
In Optimization Tool, the default in Algorithm settings > Subproblem algorithm is now Cholesky factorization, instead of Preconditioned CG = 0.
The new default can lead to slower performance for problems with high-dimensional nonlinearities. If this happens, change the default to another value such as 0 (the previous default).
The new TolConSQP
option exposes a parameter
that was fixed at eps
before. The parameter is
used in the fmincon
, fminimax
, fgoalattain
,
and fseminf
solvers.
The new default value is TolConSQP
= 1e–6.
This did not affect a vast majority of tested cases, and was beneficial
in some. If you want exactly the same behavior as before, set TolConSQP
= eps
using optimset
.
The constrviolation
field now exists in the
output structure for the fgoalattain
, fmincon
, fminimax
,
and fseminf
functions;
it measures the nonlinear constraint violation.
fminimax
now
returns the value of max(fval)
in the output maxfval
.
The iterative display of fminimax
and fgoalattain
have
changed.
The third output argument of the solver fminimax
, maxfval
,
is described in the documentation as the maximum of the objective
functions in the input fun
evaluated at the solution x
,
that is, max(fval)
. Before this release, fminimax
actually
returned the maximum of the objective functions in the reformulated
minimax problem internally constructed by the algorithm. This value
was typically very close to, but not necessarily equal to, max(fval)
. fminimax
now
returns the exact value of max(fval)
in the output maxfval
.
The iterative display for fminimax
includes
a new column with header Objective value
that reports
the objective function value of the nonlinear programming reformulation
of the minimax problem. The column header Max{F,constraints}
has
been changed to Max constraint
, and the column
now contains the maximum violation among all constraints, both internally
constructed and user-provided.
The iterative display for fgoalattain
now
shows the value of the attainment factor in the Attainment
factor
column. A new column, Max constraint
,
contains the maximum violation among all constraints, both internally
constructed and user-provided.
The Optimization Tool is a graphical user interface (GUI) for
performing common optimization tasks with the Optimization Toolbox.
Using the optimtool
,
you can do the following:
Select a solver and define your optimization problem.
Set and inspect optimization options and their default values.
Run problems and visualize results.
Import and export problem definitions, algorithm options, and results between the MATLAB workspace and the Optimization Tool.
Automatically generate M-code to capture, automate, and recreate your problem.
Access built-in help.
You can now specify the PlotFcns
option in
the optimset
function or using the Optimization
Tool for use with an Optimization Toolbox solver. With this option,
you can plot various measures of progress while the algorithm executes.
You can select from several predefined plots, or you can write your
own.
You can now specify more than one output function in the OutputFcn
option.
The output function input x
and fields in
the optimValues
structure have the following changes
that address bugs in previous releases:
residual
now returns the residual
vector for lsqnonlin
and lsqcurvefit
.
resnorm
contains the sum of squares
and has been added for lsqnonlin
and lsqcurvefit
.
The previous field fval
has been removed for these
functions.
procedure
has been removed for lsqnonlin
, lsqcurvefit
,
and fsolve
.
x
now returns the expected shape
and size for fgoalattain
and fminimax
.
The above changes to the input x
and optimValues
structure
have the following compatibility considerations in the output function:
If you have references to the residual
in
a previous version, note that the value of this field has changed
for lsqnonlin
and lsqcurvefit
.
This fixes the problem addressed by the bug report S-289285.
Any references to fval
for lsqnonlin
and lsqcurvefit
need
to be updated to resnorm
. This fixes the problem
addressed by the bug report S-289285.
Any references to procedure
for lsqnonlin
and lsqcurvefit
need
to be removed. This fixes the problem addressed by the bug report S-291974.
Previously, for fgoalattain
and fminimax
, x
returned
a column vector with an additional last element. If you have references
to the values for x
in a previous version, the
extra element must be removed and the output vector may need to be
reshaped. This fixes the problem addressed by the bug report S-315658.
You can now set the optimization option Display
to 'notify'
for
the functions fmincon
, fminunc
, fminimax
, fgoalattain
,
and fseminf
.
When Display
is set to 'notify'
,
the output is displayed only if the function does not converge.
Release | Features or Changes with Compatibility Considerations |
---|---|
R2015a | |
R2014b | |
R2014a | |
R2013b | Solvers that check initial point more carefully |
R2013a | |
R2012b | Changing default algorithms for fmincon and quadprog |
R2012a | |
R2011b | |
R2011a | |
R2010b | None |
R2010a | lsqnonneg No Longer Uses x0 |
R2009b | |
R2009a | |
R2008b | |
R2008a | |
R2007b | None |
R2007a | Changes to Outputs of Multiobjective Solvers |
R2006b | Changes to the Output Function |
R2006a | None |
R14SP3 | None |