TransWikia.com

Is there a third "version" of difference-of-means hypothesis testing?

Mathematics Asked on November 18, 2021

I’m following Schaum’s Outlines for statistics as well as taking a course and I’m getting mixed up with the way hypothesis testing is done for differences of means.

First the class described a "two-sample unpaired t-test":
$$
t = frac{(bar{x}_1 – bar{x}_2)-(mu_1 – mu_2)}{sqrt{frac{s_1^2}{n_1}+frac{s_2^2}{n_2}}}
$$

Then a "two-sample paired t-test":
$$
t = frac{bar{x}_D -mu_{test}}{frac{s_D}{sqrt{n}}}
$$

Where I assume $bar{x}_D=bar{x}_1 – bar{x}_2$ as a way to shorten what they wrote previously in the unpaired test, but the document doesn’t confirm this. I also assume $mu_{test}$ is meant to be $mu_1-mu_2$. I also found it interesting they only take about t-tests, ignoring anything with the z-score, whereas Schaum’s starts the chapter off with this for "test for difference of means":
$$
z = frac{(bar{x}_1 – bar{x}_2)-0}{sqrt{frac{sigma_1^2}{n_1}+frac{sigma_2^2}{n_2}}}
$$

I think this is because the course is focused on Python statistics, so we go straight to the more practical case where we don’t know population variance?

So I believe Schaum’s test is what’s meant to be the "two-sample unpaired t-test", but for the case where population variance is known. I’m not sure what the paired test is doing though.

Furthermore, in Schuam’s under small sampling theory, they give this for the difference of means test:
$$
t = frac{bar{x}_1 – bar{x}_2}{sigmasqrt{1/n_1 + 1/n_2}} \
sigma = sqrt{frac{n_1 s_1^2+n_2 s_2^2}{n_1+n_2-2}}
$$

I assume the zero was left out for brevity. I have no idea wht this is, and if it has anything to do with the paired test. Am I looking at essentially three tests? 1) testing if means of two distributions are difference, using $t$ if pop variance unknown and $z$ if known, 2) some sort of "unpaired" distributions and 3) whatever that small sample test is supposed to be?

One Answer

Let's go through these one at a time. The key takeaway throughout this discussion is that the choice of test statistic is a tradeoff between the distributional assumptions that can be made about the groups being compared, versus the power of the test to reject the null hypothesis. For example, you could use a nonparametric test to compare the means of two normally distributed populations with known variances, but it won't be as powerful to detect a difference compared to a two-sample $z$-test.

The first test statistic, $$T = frac{(bar x_1 - bar x_2) - (mu_1 - mu_2)}{sqrt{frac{s_1^2}{n_1} + frac{s_2^2}{n_2}}}$$ is the Welch $boldsymbol t$-test which is an approximately $t$-distributed statistic under the assumption of the null hypothesis. The degrees of freedom for this test is calculated using the Welch-Satterthwaite approxmation $$nu = frac{left(frac{s_1^2}{n_1} + frac{s_2^2}{n_2}right)^2}{frac{s_1^4}{n_1 (n_1-1)} + frac{s_2^4}{n_1 (n_2-1)}}.$$ Note that this test is applied when there is no assumption about the equality of the within-group variances; i.e., we may use this test when $sigma_1^2 ne sigma_2^2$. This is the most flexible test of location for two normally distributed groups. It does not assume their variances are known or equal, and does not require equal group sample sizes. Moreover, it is reasonably robust to deviations from normality of the groups; just like in the one-sample hypothesis test when the population is not normally distributed, the larger the sample size, the closer to asymptotic normality the sample mean becomes due to the CLT. But in the small-sample case, use of this test when normality cannot be assumed, should be reconsidered. A nonparametric test may be more appropriate.

The second test statistic, $$T = frac{bar x_D - mu_{text{test}}}{s_D/sqrt{n}},$$ is for a paired $boldsymbol t$-test. Here, $mu_{text{test}}$ is the hypothesized difference in population means, and $s_D$ is the sample standard deviation on the paired differences; i.e., $$s_D^2 = frac{1}{n-1} sum_{i=1}^n left((x_{i,1} - x_{i,2}) - (bar x_1 - bar x_2)right)^2,$$ where $x_{i,j}$ is the $j^{rm th}$ observation in group $i$ and $bar x_1 - bar x_2$ is the difference of sample means, which is equal to the sample mean of the paired differences.

This test is applicable when observations from each group can be naturally paired with each other, thus necessitating equal numbers of observations from each group. An example where such a test applies is if we are interested whether the use of a particular gasoline additive improves gas mileage. Assuming mileage is normally distributed, we collect data on a fleet of cars, running them twice: once with and once without the additive, and calculate their mileages. By calculating the difference between mileages for each car, we are in effect controlling for the between-car fuel efficiency variation. The resulting test is more powerful than a two-sample independent $t$-test for this reason.

The third test statistic, $$Z = frac{(bar x_1 - bar x_2) - 0}{sqrt{frac{sigma_1^2}{n_1} + frac{sigma_2^2}{n_2}}}$$ is a two-sample independent $z$-test of the equality of means when the within-group variances $sigma_1^2, sigma_2^2$ are known and the groups are normally distributed. In such a case, as we have discussed in another question, the within-group sample means $bar x_1, bar x_2$ are exactly normally distributed with means $mu_1$, $mu_2$, variances $sigma_1^2/n_1$, $sigma_2^2/n_2$; therefore, their difference is also exactly normally distributed: $$bar x_1 - bar x_2 sim operatorname{Normal}left(mu_1 - mu_2, frac{sigma_1^2}{n_1} + frac{sigma_2^2}{n_2}right).$$ Therefore, $Z$ is standard normal under the null hypothesis $mu_1 = mu_2$.

The fourth test statistic, $$T = frac{bar x_1 - bar x_2}{sigma sqrt{1/n_1 + 1/n_2}}$$ with $sigma$ specified as in your question, is a two-sample independent $t$-test using a pooled variance estimate. As noted in the above comment, the formula is incorrect if the sample standard deviations are calculated with Bessel's correction. This statistic has (very slightly) more power than the Welch $t$-test if the assumption that the group variances are (roughly) equal is valid.

Note: All four statistics assume normality or approximate normality of the groups. The Welch $t$-test does not assume anything else. The paired $t$-test assumes the observations are naturally paired. The $z$-test assumes the group variances are known. The pooled $t$-test assumes the group variances are roughly equal.

Answered by heropup on November 18, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP