We use the T Test to compare the means of two data sets. We begin by assuming that the means are statistically the same for both data sets. This doesn’t mean that the means of the two sample data sets we’re looking at must be exactly the same, but rather that they are close enough to each other that we can safely assume that the larger populations from which the samples we taken have essentially the same means.
So, our default hypothesis (written as H0, and referred to as the “null-hypothesis” or “h-naught”) is that means are statistically the same. Of course, we also must have an alternate hypothesis (written as H1, sometimes called “h-one”). The alternate hypothesis can take one of three forms:
- The means are not the same (this is the most general form, and the default)
- The mean of Data Set 1 is greater than the mean of Data Set 2
- The mean of Data Set 1 is less than the mean of Data Set 2
Let’s look at a specific example to illustrate how this works. A company has an assembly operation, and they would like to reduce the assembly cycle time in order to increase production capacity. Over the course of a week, they have a production engineer record the cycle time for completing 15 assemblies. They find that the average cycle time (CT) is 17.65 minutes.
They then ask the assembly team to conduct an improvement project. The team adopts several process changes that they believe will reduce cycle time. Once the changes have been tested and the whole team is trained, they begin using the new assembly process. The production engineer returns and records the cycle times for another 15 assemblies. The average cycle time is found to be 16.33 minutes. This looks like an improvement, but is it statistically significant? Another way to think about the question is, how likely is it that the old process would have produced an average that low?
To find out, the production engineer clicks the “Compare data sets” button on the SuperEasyStats ribbon and selects “T Test” from the menu of options. The engineer then sees the following dialog box:
Notice that the default hypothesis is fixed. We always assume that the means are the same unless proven otherwise. In this case, the engineer chooses as their alternate hypothesis that the mean of Data Set 1 (the “Original CT”) is larger than that of Data Set 2 (“Improved CT”). They have good reason to think this since the improvements were all made with the goal of reducing the cycle time. It’s unlikely that the cycle time increased!
The engineer won’t be willing to reject the default hypothesis unless they reach a certain confidence level that they’re different. The key question they need to answer is: how confident do they need to be?
That’s where alpha comes in. Alpha is your decision rule for rejecting the default hypothesis. If the Production Engineer wants to be 95% confident that the data sets have different means before rejecting H0, then they should set alpha to 0.05. If they want to be even more confident (say, 99%), then they should set the alpha value to 0.01. In this case the engineer wants to be 95% sure that the new cycle time is lower than the old one before accepting that as the case, so they set alpha to 0.05 (the default value).
After they’ve chosen their alpha value, they need to indicate if the data sets have equal variances. Since they don’t know for sure, they select “Not equal (or unknown)”. Once they’ve done that, they click the “Create data entry sheet” button. Below you can see the data entry sheet used by the production engineer.
In the upper left corner, the hypotheses are shown. The previously chosen alpha value is displayed below that. In the center are columns where the engineer entered the cycle times they had recorded both before and after the improvement project. To the right of those columns are summary boxes showing the mean and standard deviation of each data set.
The key result is the p-value shown in the box labels “T Test Results”. Notice that the p-value is 0.007. Since that is less than the engineer’s chosen alpha value (0.05), they have met their threshold. They can be 99.34% certain that the alternate hypothesis is true, and remember, the alternate hypothesis is that the original cycle time is larger than the improved cycle time. The assembly team has made a statistically significant improvement!
