Chapter 6: Analysing the Data Part III: Common Statistical Tests

# Special Case: The Matched Samples Design

Howell prefers to use the term 'matched samples' for the repeated measures t-test. He uses this to imply the samples providing the data are related (or correlated) in some way. They are most often related by being the same people in each treatment condition, but there is one situation in which a matched samples design is used when, in fact, separate individuals are used. Howell describes this situation on p. 187-188. Ray discusses matching as a control procedure on pages 225-230. The idea behind the matched samples design is that the advantage of greater power and economy found with repeated measures can be applied to the situation in which separate individuals are employed.

The term 'matched samples' will be reserved here for the special case of the repeated measures design in which two separate groups of individuals contribute the two sets of scores but each member of one group is matched with a corresponding member of the other group. The matching may occur on one or several variables (e.g., socioeconomic class, age, IQ). The most ideal use of this procedure occurs when identical twins are used in a research design. Each twin serves as a control for the other Ð they are therefore matched on an innumerable physical and mental characteristics. Another use is described in Howell, when husbands and wives are matched with each other. Husbands and wives are likely to be similar to each other on many attitudes and behaviours. They are supposedly not random subjects; they have selected each other because of some similarities! Also, husband's and wive's attitudes and behaviours can be more directly related. If a questionnaire item asks about satisfaction with marriage and the wife reports dissatisfaction, then the husband will also, most likely, report dissatisfaction because of a causal relationship.

Yet another situation occurs when each individual in one group is matched with an individual in the other group on the basis of pretest (or baseline) scores. For example, the person with the lowest anxiety in one group is matched (paired) with the person with the lowest anxiety in the other group. The scores from the two people are treated as if the same person produced them.

Often the matched samples design is useful when only small numbers of people are available for the research. For example, consider the situation in which you were interested in comparing the effect of a new drug for treating depression and only twenty people with clinical depression were available. If you randomly assign these twenty people to the two groups (i.e., the treatment and the control group), just by chance we might get 12 or 13 say of the most severely depressed people in one group and only 7 or 8 in the other group. The response to the drug might be most evident with severely depressed people. We might get a significant result just because more of the severely depressed people ended up in one group and not the other. If instead we ranked all 20 people in terms of depression from least severe to most severe, we could then take the two least-severe and randomly assign one to the treatment condition and one to the control condition. We then move to the next two least-severe subjects and randomly assign one of those to the treatment condition and the other to the control condition. And so move down the list of people randomly assigning one of each pair to each condition. In this way, the composition of the final two groups should be quite homogeneous in terms of the baseline (or pretest) level of depression.

The problem with using this design is that the matching variable must have a significant relationship with the dependent variable. IQ is closely related to ability to learn, therefore a study examining the effect of a new learning strategy might well use IQ as a matching variable. Level of depression is probably closely related to response to an anti-depressant drug, so this would be a suitable matching variable in this situation. If the matching variable does not have a significant relationship with the DV, the power of using this design may well be lost (because of the reduced df).