Timeline for Information criteria for ridge regression
Current License: CC BY-SA 2.5
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 16, 2010 at 14:20 | vote | accept | laxxy | ||
| Nov 16, 2010 at 14:20 | comment | added | laxxy | Thanks for the references! The situation is this: there is a model that generates data (Y and X). Under some parameter values, some X's may happen to be very close, plus some small noise, likely due to simulation error. Full-sample OLS tries to use this noise, and produces crazy estimates. However, it is almost always the best model in cross-validation. Setting the ridge parameter first and then doing cross-val. to pick components of X seems to work better, but I am not sure if there is a good way to do that. Have to look closer at lasso, first impression so far is that it overfits somewhat. | |
| Nov 15, 2010 at 3:01 | comment | added | R Hahn | OK, I misread the question -- I thought you were asking about using BIC or AIC to select the value of the ridge parameter. For doing subset selection of the predictors I'd look into the lasso estimator rather than ridge regression. There is a whole ton of literature on penalized regression, of which ridge is one flavor and lasso is another. The Elements of Statistical Learning is probably the classical text for these things. | |
| Nov 15, 2010 at 2:25 | comment | added | laxxy | Thanks! Do I understand it right that in your notation $\lambda$ is the ridge parameter ($\eta$ in the question)? I am a bit confused as to how to use these priors to construct some analogue of AIC or BIC, I am actually concerned with selecting a subset of $X$'s to use. It does look like cross-validation is the way to go, as you suggest. | |
| Nov 14, 2010 at 15:59 | history | answered | R Hahn | CC BY-SA 2.5 |