Information criteria for ridge regression - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-22T08:55:58Z http://mathoverflow.net/feeds/question/46044 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/46044/information-criteria-for-ridge-regression Information criteria for ridge regression laxxy 2010-11-14T14:26:17Z 2010-11-14T15:59:34Z <p>Hi -- is there any analogue or adjustment of, say, Schwartz Bayesian (or other) information criterion that would be applicable to model selection in ridge regression with a given ridge parameter $\eta$, i.e. $\hat{\beta}_\eta = (X'X+\eta I)^{-1}X'Y$?</p> <p>Thanks!!</p> http://mathoverflow.net/questions/46044/information-criteria-for-ridge-regression/46053#46053 Answer by R Hahn for Information criteria for ridge regression R Hahn 2010-11-14T15:59:34Z 2010-11-14T15:59:34Z <p>The ridge estimator corresponds to the posterior mean under a Normal linear regression model with a conjugate Normal-inverse-gamma prior on the regression coefficients: $\beta \mid \sigma^2, \lambda \sim \mbox{N}(0, \lambda^{-1}\sigma^2 \mbox{I})$ and $\sigma^2 \sim \mbox{IG}(a,b)$ for known hyperparameters $a$ and $b$. One may additionally put a prior distribution over $\lambda$. If you consider a discrete number of possible values for $\lambda$ then one may compute posterior probabilities for each of these values or compute Bayes factors to compare different values.</p> <p>As BIC and AIC and other such "information criterions" can be viewed as approximations to Bayes factors, this may answer your question. Usually, as you probably know, one simply checks prediction error for the different values via cross-validation (at least in prediction contexts) and selects lambda that way. </p>