Writing Vector and Matrix Objective Functions

What Are Vector and Matrix Objective Functions?

Some solvers, such as fsolve and lsqcurvefit, have objective functions that are vectors or matrices. The main difference in usage between these types of objective functions and scalar objective functions is how you write their derivatives. The first-order partial derivatives of a vector-valued or matrix-valued function is called a Jacobian; the first-order partial derivatives of a scalar function is called a gradient.

For information on complex-valued objective functions, see Complex Numbers in Optimization Toolbox Solvers.

Jacobians of Vector Functions

If x is a vector of independent variables and F(x) is a vector function, the Jacobian J(x) is

$J_{i j} (x) = \frac{\partial F_{i} (x)}{\partial x_{j}} .$

If F has m components and x has k components, J is an m-by-k matrix.

For example, if

$F (x) = [\begin{matrix} x_{1}^{2} + x_{2} x_{3} \\ \sin (x_{1} + 2 x_{2} - 3 x_{3}) \end{matrix}],$

then J(x) is

$J (x) = [\begin{matrix} 2 x_{1} & x_{3} & x_{2} \\ \cos (x_{1} + 2 x_{2} - 3 x_{3}) & 2 \cos (x_{1} + 2 x_{2} - 3 x_{3}) & - 3 \cos (x_{1} + 2 x_{2} - 3 x_{3}) \end{matrix}] .$

The function file associated with this example is:

function [F jacF] = vectorObjective(x)
F = [x(1)^2 + x(2)*x(3);
    sin(x(1) + 2*x(2) - 3*x(3))];
if nargout > 1 % need Jacobian
    jacF = [2*x(1),x(3),x(2);
        cos(x(1)+2*x(2)-3*x(3)),2*cos(x(1)+2*x(2)-3*x(3)), ...
        -3*cos(x(1)+2*x(2)-3*x(3))];
end

To indicate to the solver that your objective function includes a Jacobian, set the SpecifyObjectiveGradient option to true. For example:

options = optimoptions('lsqnonlin','SpecifyObjectiveGradient',true);

Jacobians of Matrix Functions

To define the Jacobian of a matrix F(x), change the matrix to a vector, column by column. For example, rewrite the matrix

$F = [\begin{matrix} F_{11} & F_{12} \\ F_{21} & F_{22} \\ F_{31} & F_{32} \end{matrix}]$

as a vector f

$f = [\begin{matrix} F_{11} \\ F_{21} \\ F_{31} \\ F_{12} \\ F_{22} \\ F_{32} \end{matrix}] .$

The Jacobian of F is defined in terms of the Jacobian of f,

$J_{i j} = \frac{\partial f_{i}}{\partial x_{j}} .$

If F is an m-by-n matrix, and x is a k-vector, the Jacobian is an mn-by-k matrix.

For example, if

$F (x) = [\begin{matrix} x_{1} x_{2} & x_{1}^{3} + 3 x_{2}^{2} \\ 5 x_{2} - x_{1}^{4} & x_{2} / x_{1} \\ 4 - x_{2}^{2} & x_{1}^{3} - x_{2}^{4} \end{matrix}],$

then the Jacobian of F is

$J (x) = [\begin{matrix} x_{2} & x_{1} \\ - 4 x_{1}^{3} & 5 \\ 0 & - 2 x_{2} \\ 3 x_{1}^{2} & 6 x_{2} \\ - x_{2} / x_{1}^{2} & 1 / x_{1} \\ 3 x_{1}^{2} & - 4 x_{2}^{3} \end{matrix}] .$

Jacobians with Matrix-Valued Independent Variables

If x is a matrix, define the Jacobian of F(x) by changing the matrix x to a vector, column by column. For example, if

$X = [\begin{matrix} x_{11} & x_{12} \\ x_{21} & x_{22} \end{matrix}],$

then the gradient is defined in terms of the vector

$x = [\begin{matrix} x_{11} \\ x_{21} \\ x_{12} \\ x_{22} \end{matrix}] .$

With

$F = [\begin{matrix} F_{11} & F_{12} \\ F_{21} & F_{22} \\ F_{31} & F_{32} \end{matrix}],$

and f having the vector form of F, the Jacobian of F(X) is defined as the Jacobian of f(x):

$J_{i j} = \frac{\partial f_{i}}{\partial x_{j}} .$

So, for example,

$J (3, 2) = \frac{\partial f (3)}{\partial x (2)} = \frac{\partial F_{31}}{\partial X_{21}}, and J (5, 4) = \frac{\partial f (5)}{\partial x (4)} = \frac{\partial F_{22}}{\partial X_{22}} .$

If F is an m-by-n matrix and x is a j-by-k matrix, then the Jacobian is an mn-by-jk matrix.