Interpretations among t-test, ANOVA, Tukey HSD results? - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T13:38:05Z http://mathoverflow.net/feeds/question/23331 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/23331/interpretations-among-t-test-anova-tukey-hsd-results Interpretations among t-test, ANOVA, Tukey HSD results? crippledlambda 2010-05-03T09:44:36Z 2010-08-10T17:22:16Z <p>I have one continuous variable, Variable, and two categorical variables, Factor1 and Factor2, each comprising two levels. What does it mean if </p> <p>1) According to a t-test, the difference in Variable between the two levels in Factor2 are statistically significant only for a subset of Factor1 (only within one of its levels). </p> <p>2) The pairwise comparisons according to Tukey's Honest Significant Difference test and the F-statistic for each of the categorical variables in a 2-way ANOVA both show that the influence of these categorical variables are not significant (and interactions also not significant).</p> <p>?</p> <p>This seems like an elementary textbook question but my lack of formal statistical training is beginning to show...</p> <p>Thank you in advance for your responses.</p> http://mathoverflow.net/questions/23331/interpretations-among-t-test-anova-tukey-hsd-results/31708#31708 Answer by madhadron for Interpretations among t-test, ANOVA, Tukey HSD results? madhadron 2010-07-13T14:21:19Z 2010-07-13T14:21:19Z <p>It is very often the case that some subset of your data will come out to be statistically significant by random chance. If you are running t-tests among the levels given the level of each other factor, that's four tests. Your chance of one of those four comparisons being significant at $\alpha=0.05$ is $1 - (0.95)^4$, or about 20%.</p> <p>The ANOVA, F-test, and HSD all account for this kind of multiple comparison. It's what they were designed to handle.</p> <p>It's actually quite a nice example of why multiple comparisons are important to take into account.</p> http://mathoverflow.net/questions/23331/interpretations-among-t-test-anova-tukey-hsd-results/33544#33544 Answer by PeterR for Interpretations among t-test, ANOVA, Tukey HSD results? PeterR 2010-07-27T16:36:06Z 2010-07-27T16:36:06Z <p>You might want to post this question at <a href="http://stats.stackexchange.com/" rel="nofollow">http://stats.stackexchange.com/</a> Went public beta yesterday</p>