Question

Is there a general method of determining the line of best fit (using the principle of least squares or any other principle) for any given set of data points$? If there is no general method, what is/are the next best options?

This problem is motivated by the difficulty in deciding which curve will best fit a given data set. If the data is roughly linear, I can use linear regression. If the data shows a quadratic behavior, I can guess that a quadratic curve will best fit the data and according I will try to find the best quadratic fit. But if the data show no particular trend of if it has a trend which I am not able to determine by simple observation, it is difficult to guess which model will best fit the data. For example, using linear or quadratic regression on a data that has the hidden pattern $y=x^{2.5}\ln x$ (which is difficult to guess) is not effective.

Hence I am looking for general a method of regression using least squares or any other principle that will work for all kinds of data.

Answer 1

This might be more appropriate as a comment, but I lack the rep to leave one. What you might be looking for is maximum entropy curve fitting, which, in some sense, produces the best-fitting curve under the least amount of assumptions (in an information theory sense). I don't have good references at hand. Here are a couple of presentations applying the method to data in cold atoms.