Borrar filtros
Borrar filtros

Hessian in mlecov compared to hessian from fmincon

5 visualizaciones (últimos 30 días)
zym
zym el 27 de Ag. de 2022
Comentada: Bruno Luong el 27 de Ag. de 2022
Hi, I am doing custom MLE estimation using fmincon. I want to compute asymptotic covariance matrix of MLE. To do so I could use mlecov or Hessian from fmincon. I was wondering if the hessian is estimated in mlecov in the same way as in fmincon? If not, what are possible advantages of mlecov?
Since I am running a custom problem, mlecov gives me an issue about positive definiteness. I am thinking to opt for the hessian as coming from the fmincon. However, the documentation suggest that the hessian output from fmincon is not precise https://nl.mathworks.com/help/optim/ug/hessian.html . What are other possible options? I identify three:
  1. Use the maximised value of parameters from fmincon and plug it in fminunc to obtain the hessian, which seems to be correctly estimated there. Update: I still find that the Hessian is not-positive definite
  2. Correct the hessian in mlecov to be positive definite (though I am not sure if it is a reliable procedure) Update: In the comment below, Matt J suggests a great function to that.
  3. Drop using fmincon. Since my constraints are only non-negativity of variance, transform parameters and just use fminunc. Update: Hessian from fminunc still comes out to be non positive definite. However, mlecov does the magic. Once I plugin maxima from fmincon in the yet another custom function where parameters are transformed, such the problem is unconstrained, I find that mlecov is able to give me a positive-definite Hessian.
Did any of you happenned to have the same issue?
Thanks for your help. Regards!
  2 comentarios
Matt J
Matt J el 27 de Ag. de 2022
Editada: Matt J el 27 de Ag. de 2022
You could also just compute the Hessian yourself, analytically.
zym
zym el 27 de Ag. de 2022
Editada: zym el 27 de Ag. de 2022
Thanks, it is a good point. But my problem is too complicated for the analytical solutions. Includes Kalman filter etc...

Iniciar sesión para comentar.

Respuestas (1)

Matt J
Matt J el 27 de Ag. de 2022
Editada: Matt J el 27 de Ag. de 2022
I was wondering if the hessian is estimated in mlecov in the same way as in fmincon?
Probably, but even if not, the Hessian inversion will be sensitive to errors from finite difference computation.
You could try nearestSPD to find the nearest positive semi-definite matrix to whatever fmincon gives you,
Be mindful of a a few things, though,
(1) A positive semi-definite matrix is still not positive definite.
(2) The theory of approximating MLE covariance with Hessians is only valid when no constraints are active at the solution.
  4 comentarios
zym
zym el 27 de Ag. de 2022
Thanks for this. However, trust region algorithm needs analytical gradient, which I do not have.
To obtain asymptotic covariance matric, mlecov computes Hessian using final differences. Inside the function, they write:
% Compute pure and mixed second order central differences of the
% log-likelihood function. H is always a square matrix, regardless of
% the shape of params. nH is the negative hessian, because loglikefun
% returns a negative log-likelihood.
Bruno Luong
Bruno Luong el 27 de Ag. de 2022
If that is the case, I would trust the Hessian returned by mlecov much more.
The Hessian used and returned by optimlization algorithms are designed for ... optimization, meaning the main purpose is to find a correct descend direction and not too costly. So I would not use it to make any further fine analysis.

Iniciar sesión para comentar.

Categorías

Más información sobre Solver Outputs and Iterative Display en Help Center y File Exchange.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by