The “Multiple data sets” button allows you to compare the means of between 3 and 10 data sets. This tool is often called “1-way ANOVA” because the data sets should differ in only one way (and “ANOVA” is short for ANalysis Of VAriance). For example, perhaps your company has a process that is performed by different people using the same standard work, materials, and procedures. The one difference is the person doing the work. If you wanted to measure, say, their average time to complete the process, you could use 1-way ANOVA to do it. The example on the right shows how that works.
The first step is to choose your alpha value. In traditional hypothesis testing, we begin by assuming that the characteristic we’re assessing (average time in this example) is the same for all operators. Our default hypothesis (H0, also known as the “null-hypothesis” or “h-naught”) is that each operator completes the process in the same average time. We won’t be willing to reject that default hypothesis unless we reach a certain confidence level that at least of the operators has an average time that’s statistically different from the others. The key question you need to answer is: how confident do you need to be before you’re willing to accept that there’s a difference? That’s where alpha comes in. Notice the cell in the upper left labeled “alpha”. That’s where you set your decision rule. If you want to be 95% confident that the average times are different, then set alpha to 0.05. If you want to be even more confident (say, 99%), then set alpha to 0.01. It’s up to you!
Next, since there are five operators, select that number in the “Groups to compare” field. Then simply label your columns and enter the sample values for each operator.
The p-value is the result of a statistical calculation meant to determine how likely it is that that the samples for these five operators all come from the same distribution withe same mean. If the p-value ends up being lower than your chosen alpha value (i.e., your decision rule), then you can confidently reject H0 and accept the alternative hypothesis (known as H1 or “h-one”) that not all of these operators have the same average time. In this case, the p-value is, in fact, lower than your chosen alpha value. Based on your that, you can reject the default hypothesis and accept the alternative hypothesis (that not all the operators have the same average time) with the confidence shown (96.72% in this case).
Note that the use of this test assumes that each data set has approximately the same variance, which you can see in the calculated values in the table at the top.
Copyright 2020 David Margil