5
$\begingroup$

I was recently trying to understand generalized linear models (GLMs) and after investing quite a few days, it still hasn't dawned on me what the fundamental benefit of the framework is. Normally, I am used to results like guarantees of convergence, limits for error etc, but all that seems to be missing here.

There is a common framework with underlying distribution, regressors/predictors linear in the coefficients, link functions and finally MLE but it seems to be branching off very quickly into the various subclasses, which each need a separate algebraical and numerical treatment.

So can anyone point me towards what is "general" about the GLMs and what is the benefit of that?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ This is sort of implicit in @kjetilbhalvorsen's answer, so I'll leave it as a comment rather than an additional answer. A big part of the motivation comes from engineering: the numerical algorithms used to fit GLM's to data are pretty similar across the whole class of models, so you can decouple the hardcore software engineering from the downstream applications. As a practitioner, this is a big deal: any time I can express a model as a GLM I know I won't have to worry too much about running the model at scale. $\endgroup$
    – Paul Siegel
    Commented Sep 15, 2020 at 19:44
  • 2
    $\begingroup$ Relatedly: I read somewhere that among the thousands of machine learning models Google has in production, something like 80% of them are GLM's. They are kind of the unsung heros of the machine learning world. $\endgroup$
    – Paul Siegel
    Commented Sep 15, 2020 at 19:46

1 Answer 1

Reset to default
8
$\begingroup$

What are the benefits of a unified framework? You are right that we are rapidly going into some much used special cases line logistic regression or Poisson regression, but there is still benefit in having a common framework.

A lot of information can be found at Cross Validated, see as a start https://stats.stackexchange.com/questions/104399/why-do-we-use-glm

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thank you, that confirms my intution and will lead to me looking more into the practical applications of the subclasses. Is there any reason that you mention IRLS instead of Gradient Descent? Does it have preferable features w.r.t. GLMs? $\endgroup$
    – Manuel Huppertz
    Commented Sep 15, 2020 at 23:08
  • $\begingroup$ Nevermind, figured it our on my own. The advantage is the absence of the gradient. $\endgroup$
    – Manuel Huppertz
    Commented Sep 15, 2020 at 23:18
  • $\begingroup$ IRLS is what is traditionally used, it amounts (in simple unmodified form) to a version of Newton's method. There is no reason today to not use other optimization methods ... but IRLS is very easy to implement if you have a function for weighted least squares, so was very well fitted for the early 1970's ... $\endgroup$
    – kjetil b halvorsen
    Commented Sep 16, 2020 at 0:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .