Finding correlation in large data, non-numeric sets - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-24T06:18:12Z http://mathoverflow.net/feeds/question/6002 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/6002/finding-correlation-in-large-data-non-numeric-sets Finding correlation in large data, non-numeric sets Frank Meulenaar 2009-11-18T19:56:41Z 2009-11-18T21:25:30Z <p>Suppose I collect a lot of data from a group of persons, like</p> <ul> <li>their height</li> <li>their weight</li> <li>color of eyes (chosen from eg the four categories blue/brown/black/other)</li> <li>sex</li> <li>day of the week the measurement was done</li> </ul> <p>and I want to find which of the above data correlates with height.</p> <p>For weight, I can image how that goes: just make the (x,y) set and measure stuff like Pearson's correlation. But I get stuck for things like 'colors of eyes' where there's no natural way to order them, or binary data like sex. Also for day of the week (where I'd like to find no correlation at all) but I wouldn't know how to order them.</p> <p>How do I do this?</p> http://mathoverflow.net/questions/6002/finding-correlation-in-large-data-non-numeric-sets/6011#6011 Answer by Kim Greene for Finding correlation in large data, non-numeric sets Kim Greene 2009-11-18T20:32:17Z 2009-11-18T20:32:17Z <p>I think sex can be written as a yes/no question. Female = 1. Male = 0.</p> <p>You can extend this to eye color. Blue eyes: yes/no. Green eyes: yes/no. Brown eyes: yes/no. One variable like eye color becomes three variables that are either 0 or 1. This gives you a lot of variables but you can do numerical things with them.</p> <p>Sometimes it makes sense to look at these variables as the probability of being male, having green eyes or whatever.</p> http://mathoverflow.net/questions/6002/finding-correlation-in-large-data-non-numeric-sets/6016#6016 Answer by Jon Awbrey for Finding correlation in large data, non-numeric sets Jon Awbrey 2009-11-18T21:18:28Z 2009-11-18T21:25:30Z <p>There are lots of textbooks under headings like "Categorical Data Analysis", "Non-Parametric Statistics", <i>et cetera</i>, and all the standard statistical packages &mdash; GenStat, SAS, SPSS, and other names that will make me sound real ancient if I mention them &mdash; will have lots of practical advice about their usage on this kind of data.</p>