Standard errors from hessian using fminunc/fmincon

I'm optimizing a gaussian likelihood and using the hessian output from fminunc/fmincon to find standard errors. While this works fine in the unconstrained case, the standard errors become significantly smaller by an order of magnitude even when a single constraint is imposed on one of the statistically insignificant parameters.

Any idea why this is happening? I've attached the likelihood file for reference.

Answer 1

[Alan Weiss]

You might not have read the documentation of the Hessian output for fminunc and fmincon. It explains that the Hessian output for fmincon can be inaccurate. So if you find it is not working well, that is, unfortunately, the documented behavior.

Also, it is usually unclear what a standard error might mean in a constrained case, but that is something else.

Alan Weiss

MATLAB mathematical toolbox documentation

[V B]

Thanks for the response! I had come across that and saw that the hessian is only approximated inasmuch as it serves to find an optimal solution; but using the quasi-Newton algorithm for fminunc I figured this wouldn't be a problem.

Answer 2

[Matt J]

Sometimes a constrained problem can be reformulated as an unconstrained one, e.g., in one dimension

   min. x
   s.t. x>=0

is equivalent to

   min. y^2

once you make the transformation of variables x=y^2.

Once you've solved the constrained version, you might be able to use the unconstrained, transformed problem to analyze the standard error. So, in the above case, once I have the standard error for the unconstrained variable y, I could presumably propagate that through the transform x=y^2 to get the uncertainty of x.

[V B]

Thanks for the response! I will see if my problem can be reformulated like this, though I am actually interested in simple zero restrictions.

[Matt J]

If you're just setting select unknowns to zero, they are no longer unknowns, and your constraints are trivial. You should remove the zeroed variables from the problem and work with the remaining unknowns as an unconstrained problem.