Visualizing Orthogonal Polynomials - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-22T23:03:52Z http://mathoverflow.net/feeds/question/38864 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/38864/visualizing-orthogonal-polynomials Visualizing Orthogonal Polynomials jd long 2010-09-15T20:11:10Z 2010-09-16T00:35:57Z <p>Recently I was introduced to the concept of <a href="http://en.wikipedia.org/wiki/Orthogonal_polynomials" rel="nofollow">Orthogonal Polynomials</a> through the poly() function in the R programming language. These were introduced to me in the concept of polynomial transformations in order to do a linear regression. Bear in mind that I'm an economist and, as should be obvious, am not all that smart (choice of profession has an odd signaling characteristic). I'm really trying to wrap my head around what Orthogonal Polynomials are and how, if possible, to visualize them. Is there any way to visualize orthogonal polynomials vs. simple polynomials? </p> http://mathoverflow.net/questions/38864/visualizing-orthogonal-polynomials/38866#38866 Answer by Piero D'Ancona for Visualizing Orthogonal Polynomials Piero D'Ancona 2010-09-15T20:26:10Z 2010-09-15T20:26:10Z <p>Start <a href="http://dlmf.nist.gov/18.4" rel="nofollow">here</a></p> http://mathoverflow.net/questions/38864/visualizing-orthogonal-polynomials/38877#38877 Answer by Tom Smith for Visualizing Orthogonal Polynomials Tom Smith 2010-09-15T21:22:06Z 2010-09-15T21:22:06Z <p>I think of the space of polynomials on R as a set of graphs arranged round the real line like the pages of a book round the axis. Polynomials which have almost the same graph are close to each other; then orthogonal polynomials are those which fall at right angles in the picture, and the linear combinations generate the space just as the basis vectors of $\mathbb{R}^n$ generate $\mathbb{R}^n$.</p> http://mathoverflow.net/questions/38864/visualizing-orthogonal-polynomials/38881#38881 Answer by Helge for Visualizing Orthogonal Polynomials Helge 2010-09-15T21:38:16Z 2010-09-15T21:38:16Z <p>I am not sure of your math-background, so I am trying to keep it simple, without oversimplifying some ideas. First off polynomials are nice for various reasons, e.g. a polynomial of degree $n$ has at most $n$ zeros. However, there are still many polynomials and it makes sense to choose VERY nice ones: orthogonal polynomials.</p> <p>To choose orthogonal polynomials, one has a problem at hand, which comes with a way to measure functions $f: \Bbb R \to\Bbb R$ by an expression of the form $$\mathcal{E}(f) = \int_{-\infty}^{\infty} f(x)^2 w(x) dx,$$ where $w(x) > 0$ is a weight that satisfy $\int w(x) dx = 1$. One should think of $\mathcal{E}$ as an energy.</p> <p>Now the orthogonal polynomial of degree $n$ can be defined as the polynomial $P_n(x) = x^n + a_{n-1} x^{n-1} + \dots + a_1 x + a_0$, where $a_{n-1}, \dots, a_0$ are real numbers, that minimizes $\mathcal{E}(P_n)$. It is this minimization property that is responsible for some of the power of orthogonal polynomials.</p> <p>At this point let me also say that it is through the weight $w$ that your problem enters the definition of orthogonal polynomials. And that one also has orthonormal polynomials, which satisfy $\mathcal{p_n} = 1$. These are given by $p_n = \frac{1}{\sqrt{\mathcal{E}(P_n)}} P_n$.</p> http://mathoverflow.net/questions/38864/visualizing-orthogonal-polynomials/38900#38900 Answer by J. M. for Visualizing Orthogonal Polynomials J. M. 2010-09-16T00:35:57Z 2010-09-16T00:35:57Z <p>Helge presented the <em>continuous</em> case in his answer; for the purposes of data fitting in statistics, one usually deals with <em>discrete</em> orthogonal polynomials. Associated with a set of abscissas <code>$x_i$</code>, <code>$i=1\dots n$</code> is the discrete inner product</p> <p><code>$$\langle f,g\rangle=\sum_{i=1}^n w(x_i)f(x_i)g(x_i)$$</code></p> <p>where <code>$w(x)$</code> is a weight function, a function that associates a "weight" or "importance" to each abscissa. A frequently occurring case is one where the <code>$x_i$</code> are equispaced, <code>$x_{i+1}-x_i=h$</code> where $h$ is a constant, and the weight function is <code>$w(x)=1$</code>; for this special case, special polynomials called Gram polynomials are used as the basis set for polynomial fitting. (I won't be dealing with the nonequispaced case in the rest of this answer, but I'll add a few words on it if asked).</p> <p>Let's compare a plot of the regular monomials $x^k$ to a plot of the Gram polynomials:</p> <p><img src="http://i53.tinypic.com/jsfwc6.png" alt="monomial versus Gram"></p> <p>On the left, you have the regular monomials. The "bad" thing about using them in data fitting is that for $k$ high enough, $x^k$ and $x^{k+1}$ are nigh-indistinguishable, and this spells trouble for data-fitting methods since the matrix associated with the linear system describing the fit is dangerously close to becoming singular.</p> <p>On the right, you have the Gram polynomials. Each member of the family does not resemble its predecessor or successor, and thus the underlying matrix used for fitting is a lot less likely to be close to singularity.</p> <p>This is the reason why discrete orthogonal polynomials are of interest in data fitting.</p>