go to the School of Psychology home page Go to the UNE home page
Chapter 7 - Analysing the Data Part IV - Analysis of Variance Chapter 1 - Behavioural Science and research Chapter 2 - Research Design Chapter 3 - Collecting the Data Chapter 4 - Analysing the Data Part I - Descriptive Statistics Chapter 5 - Analysing the Data Part II - Inferential Statistics Chapter 6 - Analysing the Data Part III - Common Statistical Tests Probability Sampling distributions Steps in hypothesis testing Type I and Type II decision errors Power Bonferroni Confidence Intervals Readings and links


Chapter 5: Analysing the Data
Part II : Inferential Statistics


What alpha-level?

One of the key concepts in hypothesis testing is that of significance level (or, equivalently the alpha (alpha ) level) which specifies the probability level for our evidence to be an unreasonable estimate. By unreasonable, we mean that the estimate should not have taken its particular value unless some non-chance factor(s) had operated to alter the nature of the sample such that it was no longer representative of the population of interest. The researcher has complete control over the value of this significance level. While we recommend a standard, or decision criterion, of alpha = 0.05, ( for two-tailed tests), you should be cautious about the blind adoption of this level. The alpha -level should be considered in light of the research context and in light of your own personal convictions about how strong you want the evidence to be, before you will conclude that a particular estimate is reasonable or unreasonable. In some exploratory contexts (perhaps, in an area where little previous research has been done), you might be willing to be more liberal with your decision criterion and relax the level of significance to 0.10 or even 0.20 - thus, less extreme values of a statistic would be required for you to conclude that non-chance factors had operated to alter the nature of the sample. On the other hand, there are research contexts in which one would want to be more conservative and more certain that an unreasonable estimate had been found. In these cases, the significance level might be lowered to 0.001 (0.1%) where more extreme values of a statistic would be required before non-chance factors were suspected

You need to examine the SPSS output usually under headings such as "Sig." or "Two-tailed Sig.", or "Prob." for the probability (or "p-value") of your results being a real difference or a real relationship. This can then be the probability you quote as being the "level of significance" associated with your results. Note that we normally need this p-value to be less than or equal to 0.05 to make the claim of ‘significant’. It is then up to your discussion to explain/justify/interpret this level of significance to your reader. As a general guide, treat the p-value as a measure of the confidence or faith you can have in your results being real (and not being due to chance fluctuations in sampling). When evaluating your test statistic (Step 4) take into consideration the points raised in the preceding paragraph.

The alpha -level is the probability or ‘p-value’ you are willing to accept as significant. Ideally, this alpha -level. The alpha -level can also be interpreted as the chance of making a Type I error.




© Copyright 2000 University of New England, Armidale, NSW, 2351. All rights reserved

UNE homepage Maintained by Dr Ian Price
Email: iprice@turing.une.edu.au