Jon Schutz Technical Notes and Recommendations

Determining whether two means come from the same distribution

November 8, 2007 at 5:00 pm · Filed under Statistics

Useful when: you have two sets of measurements, and want to know if there has been a shift in mean value.

Use Student’s t-test for significantly different means.

Sample variance for sample A:

$Formula does not parse: s_A^2 = \frac{1}{N_A – 1}\sum_{i}(a_i – \bar{a})$

Similarly for sample B. Then

$Formula does not parse: t = \frac{\bar{a} – \bar{b}}{s_D}$

where

$Formula does not parse: S_D = \sqrt{ \frac{(N_A – 1)s_A^2 + (N_B – 1) s_B^2}{N_A + N_B – 2} (\frac{1}{N_A} + \frac{1}{N_B}) }$

Finding the significance level

Use the t-distribution with $Formula does not parse: {N_A + N_B – 2}$ degrees of freedom to compute the significance level, which is the probability that $|t|$ could be larger, by chance, for distributions of equal means. Thus a significance of 0.05 suggests that the means are different with 95% confidence.

Testing against a significance level

For e.g. a test with 95% confidence level, find the threshold value of t at 0.05 from the t-distribution. If the computed t exceeds the threshold, the means are considered different to that level of confidence.

Code pointers

Octave – t_test

Perl – Statistics::TTest

Spreadsheet – TTEST, TDIST, TINV

References:

http://en.wikipedia.org/wiki/Student’s_t-test

Numerical Recipes

Permalink

Bookmark this on Delicious