<b>"Linear regression"</b>

Statisticians know better than to think that the term "linear regression" is so called because one is fitting a straight line. Often in linear regression one fits a parabola or a sinusoid or some other sort of functions via ordinary least squares, and the word "linear" is just as apt as when one fits a straight line, and is _more_ appropriate than it would be if "linear regression" meant fitting a straight line, since, it "linear" referred to the function one is fitting rather than to how it is done, then the word "affine" would be more appropriate.

So where is the "other" definition --- the one according to which it does mean fitting a straight line? One example is when an astronomer taught a course in a statistics department on applications of statistics to astronomy, and reserved the word "linear" for fitting a straight line. Respectable scientists from outside of statistics do that.

Consider the model
$$
Y_i = a + bx_i + cx_i^2 + \text{error}_i,
$$
versus the model
$$
Y_i = a + b \cos x_i + c\sin x_i + \text{error}_i,
$$
where the ordinary least-squares estimates are $\hat a, \hat b, \hat c.$

The mapping
$$
\left[ \begin{array}{c} Y_1 \\ \vdots \\ Y_n \end{array} \right] \mapsto \left[ \begin{array}{c} \hat a \\ \hat b \\ \hat c \end{array} \right] 
$$
(with the $x$s fixed) is <b>linear</b>.

Contrast this with the model
\begin{align}
& \operatorname{logit} \Pr(Y_i=1) = \operatorname{logit}( 1- \Pr(Y_i = 0) ) = a + bx_i, \\[8pt]
& \text{where } \operatorname{logit} p = \log \frac p {1-p} \text{ for } 0<p<1.
\end{align}
(long "o", soft "g")

Here the usual estimates $\hat a, \hat b$ are not found by least squares, but rather
\begin{align}
& \left(\hat a, \hat b \right) = \operatorname*{argmax}_{a,b} \Pr( Y_1=y_1\ \& \cdots \&\ Y_n=y_n\mid a,b) \\[8pt]
= {} & \operatorname*{argmax}_{a,b} \prod_{i\,:\,y_i\,=\,1} \operatorname{logit}^{-1}(a+bx_i) \times \prod_{i\,:\,y_i\,=\,0} \left( 1-\operatorname{logit}^{-1} (a+bx_i) \right).
\end{align}
This is one of many instances of nonlinear regression.