Problematic SE with fminunc

Hi I am trying to estimate a discrete choice mixed (logit) model.

The mle (via maximum simulated lik) converges but my standard errors are imaginary. (with fewer simulations I get NaN sometimes).

I use: ihess=inv(hessian); stderr=sqrt(diag(ihess));

The hessian has some columns of zeroes and also negative numbers.

My questions are: 1. What could be causing the imaginary errors? 2. Is there some fix?

[John D'Errico]

Sorry. I was deleting so many spams, that when I looked at your question because it was of interest to me, I deleted it by accident, out of habit.

Answer 1

[John D'Errico]

If the hessian has columns of zeros, then it is only semi-definite. NOT positive definite. So it is not invertible.

Negative numbers in a Hessian are NOT a problem, as long as the MATRIX as a whole is positive definite (think positive eigenvalues.) You don't have that.

There are linear combinations of your variables that yield the same MLE function value, at least a one-dimensional subspace of your parameters has that property.

The EIG or SVD decomposition of your hessian will tell you what that subspace is.

[Matt J]

Possibly, the columns of zeros might really be zeros only to 4 decimal places (MATLAB's default display precision). If so, the hessian might be invertible, but numerically negative definite, so that

stderr=sqrt(diag(ihess))

yields the imaginary values mentioned.

[Deepiks]

Thanks, John.

Can I also give up on trying to get hessian and just bootstrap?

[John D'Errico]

Matt is probably correct, that those zeros are not truly zeros, but tiny enough to display as zero compared to the other elements of the array. This would be a common thing for a MLE problem.

A bootstrap would certainly be a common approach. However, if the hessian is numerically singular, it tells me that any solution in that nullspace will be as good as any other. So if you ran the process over and over again with different starting values, then ANY point from that infinite family of solutions may result. This is what the singularity tells you.

You can get this same information from the Hessian that you already have. The eigenvectors that correspond to the zero/near zero eigenvalues define that subspace for you. And the non-zero eigenvalues can be used to define the spread outside of the nullspace.

Regardless, I don't know that any of this will be that useful, if you are looking for parameter variances.