I would like to know if there are any standard techniques (that I don't know about) to solve the following problem.

Suppose we have $n$ variables, $\mathbf{q} = (a_1, a_2, \ldots, a_n)$, but not all of them are independent. For example, the values could be determined by only a single variable, e.g. $(x, x^2, x^3)$, or only two, e.g. $(x, y, x^2, xy, y^2)$.

Now suppose we have some measurement data for $\mathbf{q}$, which might have small errors (i.e. the relationship between the variables is valid only to some error). How can we find how many of the $n$ variables are independent of each other?

If the question is not clear, please ask. I see that there are problems, e.g. in $(x, x^2, x^3)$, $x$ can determine the value of the rest of the variables, but $x^2$ cannot (if there are negative values). Nevertheless any suggestions are most welcome. I am only interested in the number of independent variables for now, not the nature of relationship between them.

Note: I know about techniques in the case when the relationship between them is linear. I am interested in the non-linear (but continues) case now.

Note 2: Please help tag the question appropriately...


Another way to put it:

I have some points in an $n$-dimensional Euclidean space. The points lie very close to a $k$-dimensional surface. How can I estimate the value of $k$ if I know the coordinates of the points?


There should be quite an extensive literature on this type of problem. A quick Google search turned up these papers: http://www.cs.bu.edu/techreports/pdf/2011-012-intrinsic-dimension-clustering.pdf, http://www.princeton.edu/~wbialek/our_papers/chigirev+bialek_04.pdf

This, and further references given there, should get you started on a more exhaustive search.

  • $\begingroup$ @Michael, thank you, this is useful! I didn't react to your answer up til now because I was travelling. $\endgroup$ – Szabolcs Horvát Jun 13 '11 at 14:04
  • $\begingroup$ @Michael, what keywords did you use to search for this? $\endgroup$ – Szabolcs Horvát Jun 13 '11 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.