show/hide this revision's text 2 s/changes/changing argh

If you are committed to linear regression, there are two choices--regularization, and changes changing the loss function (linear regression usually means least squares, which (without regularization) can be solved conveniently with a line of matrix manipulation in matlab).

As far as regularization goes, the two techniques which get lots of attention are ridge regression (l2-regularization) and lasso (l1-regularization). These days, l1-regularization gets more attention due to connections to sparsity and also its use in compressed sensing.

If you don't have many variables, i.e. model complexity isn't the problem and you just have some outliers really messing with the hyperplane, you can use a more insensitive loss function, for instance absolute loss or huber loss (huber loss is similar to absolute loss in terms of sensitivity to outliers, but is also differentiable).

show/hide this revision's text 1

If you are committed to linear regression, there are two choices--regularization, and changes the loss function (linear regression usually means least squares, which (without regularization) can be solved conveniently with a line of matrix manipulation in matlab).

As far as regularization goes, the two techniques which get lots of attention are ridge regression (l2-regularization) and lasso (l1-regularization). These days, l1-regularization gets more attention due to connections to sparsity and also its use in compressed sensing.

If you don't have many variables, i.e. model complexity isn't the problem and you just have some outliers really messing with the hyperplane, you can use a more insensitive loss function, for instance absolute loss or huber loss (huber loss is similar to absolute loss in terms of sensitivity to outliers, but is also differentiable).