I was thinking about the problem of overfitting data. Suppose you have a hundred data points sampled from an unknown function (call this the training set). You could try fitting a hundred-dimensional polynomial to these points, but even though it fits all the sample points exactly, it probably wouldn't do a good job of fitting other points sampled from the same function (the test set). This is called overfitting the training set. So suppose you were to do this instead: choose ten points from the training set, and fit a ten-dimensional polynomial to them. Then do it again, with another ten random sampled points. Get a bunch of polynomials this way, and take an average of their predictions. Wouldn't that do a better job of avoiding overfitting? Surely someone else has had this idea before. I'm looking for what it is called, so I can read up on it.