Chapter 6: Analysing the Data
Making the decision
Now that we have the probability of the difference, given the null hypothesis (p[D|H]), we are prepared to make a decision about the population mean (that is, the entire department's publication record) from information we've obtained from a random sample of that population. We were interested in testing the null hypothesis assumption that the mean number of publications by UNE psychology department academics is equal to the national average of 8. In a random sample of 5 members of the department, we found a mean of 9.4. We also know that the probability of finding a difference this large or larger in either direction of 8 is 0.729 (or 72.9%) when randomly sampling 5 from this population if the null hypothesis is true. Now what?
In this case, p is greater than .05. So we don't reject the null hypothesis. The probability of .729 means that we'd expect to find a sample mean that deviates at least 1.4 from the null hypothesised value of 8 about 3 in 4 times (which is p = .75) we take a random sample of size 5 from this population. The obtained difference simply isn't sufficiently rare enough for us to rule out "chance" or sampling error as the explanation for the difference between the sample mean and the null hypothesised value of the population mean of 8. So we conclude that the psychology academics at UNE publish about the same as the national average: 8 times per staff since 1974. Had p been less than alpha (.05 here), we would have rejected the null hypothesis. When the result leads us to reject the null hypothesis, we claim the result is statistically significant.
Finally, two important points. First, notice that we accomplished this inference relying only on information from the sample—the sample mean, the sample standard deviation, and the sample size—and with a little knowledge about statistical theory and how "chance" influences the sample mean. We didn't need to know anything about the actual population. Second, many people think that if p<.05 then the probability that the null hypothesis is true is less than .05. But this is not correct. The p value is the probability of the obtained deviation or one more extreme occurring if the null hypothesis is true. We actually can't compute the probability that the null hypothesis is true without additional information.
One-tailed tests. In this example I've illustrated a two-tailed or non-directional hypothesis test. We have all the information we need in Output 6.2 had we conducted a one-tailed test. Notice that the sampling distribution of t is symmetrical (see Figure 6.7). The hashed area to the right of 0.372 is equal to the area to the left of -0.372. Because the p value is the sum of the proportions representing these areas, it makes sense that a one-tailed p value is derived by cutting the two-tailed p value in half. So if the alternative hypothesis was H1: m > 8, the one-tailed p value would be one half of 0.729 or about 0.365. This is still above the .05 level adopted in the first place, so had we conducted a one-tailed test, we still wouldn't reject the null hypothesis.
Critical Values. Our decisions are based on p values—the probability of the obtained difference between the sample result (the mean in this case) and the null hypothesis (or one more extreme) if the null hypothesis is true. Good statistical analysis programs will provide the p value for a null hypothesis test. Tables of critical t-values can also be obtained in most statistics texts just like we did for correlation. These critical values are sometimes important such as when constructing confidence intervals. However, for our purposes they are rarely necessary. So, we will not go through any further examples of using tables of critical values..
The five steps of hypothesis testing for the one sample t-test will be briefly presented for the problem of determining if the average number of publications for the entire staff at UNE School of Psychology differ from the national average of 8. That is, all 17 observations now form a sample instead of a population.
Step 1: State hypotheses.
Note, the null hypothesis nominates a specific value. Often this is zero, but it need not be. The alternative is a two-tailed test. The alpha level required for significance is .05
Step 2: Check assumptions
The assumptions for the one-sample t-test are
Measurements are interval scale or better
Observations are independent of each other
The scores in the parent population are normally distributed.
Number of publications is a ratio variable. The observations are probably not independent if some staff members have joint publications. The distribution of scores also does not look very normal! There are some very extreme scores. Given these cautions about the assumptions being met, we will interpret the final p-value cauriously.
Step 3: Calculate the test statistic
The second example compares the sample means of the 17 staff with the national average of 8.
Output 6.4 Compare means … One-Sample T Test
Step 4: Evaluate the statistic
Here, we focus on the "Sig. (2-tailed)" value. This is .497. Because it is not less than .05, we treat the result as not significant. We therefore have no reason to reject the null hypothesis. The summary statement is t(16) = .694, p > .05.
Step 5: Interpret the result
First, a statement such as
Second, we interpret this as indicating that the staff at UNE produce about the same number of publications as the national average. So despite being considerably over worked, we still manage to get enough papers out on average. Of course, there is quite an amount of variation in the scores!
© Copyright 2000 University of New England, Armidale, NSW, 2351. All rights reserved
Maintained by Dr Ian Price