MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In the simplest logistic regression we model the (logit) probability of a 0/1 valued outcome as a (linear) function of some given 0/1 valued predictor (dummy) variables encoding membership in certain discrete categories (gender, blood type, etc).

The parameters of the model (coefficients of the linear function) for the variables representing membership in just one of the categories are mutually orthogonal, in the sense that the relevant terms in the Fisher information matrix are zero (the entry at the row for blood type A and column for blood type AB, will be zero). They are not orthogonal to the variables representing other categories.

Generally the parameter estimates will be asymptotically normal. THE QUESTION: In this multivariate normal distribution, ** are the parameters within a single category distributed independently **? In other words, is the estimation of the "blood type A" independent of the estimate of the "blood type O" or the "blood type AB" coefficients?

If the answer is No, are there other useful consequences of the partial orthogonality?

share|cite|improve this question
Orthogonal parameters are nice from computational point of view. If your Fisher Information matrix is well conditioned, then the Hessian of (conditional) log-likelihood will be well-conditioned as well, and gradient descent will converge fast – Yaroslav Bulatov Aug 27 '10 at 3:21

Well, the variance of the maximum likelihood estimator es approximated by the inverse of the information matrix, so just compute that inverse and see?

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.