go to the School of Psychology home page Go to the UNE home page
Chapter 7 - Analysing the Data Part IV - Analysis of Variance Chapter 1 - Behavioural Science and research Chapter 2 - Research Design Chapter 3 - Collecting the Data Chapter 4 - Analysing the Data Part I - Descriptive Statistics Chapter 5 - Analysing the Data Part II - Inferential Statistics Chapter 6 - Analysing the Data Part III - Common Statistical Tests Operationalism Experimental and non-experimental designs Internal and external validity Between groups Vs repeated measures designs Ethical issues


Chapter 2: Research Design



Before a study can begin, a fundamental question must be dealt with: How do I measure what I am interested in studying? It is all well and fine to hypothesise a relationship between similarity and attraction, or between a therapeutic program and mental illness, but your ability to provide evidence for such hypotheses will remain at a stand-still until you are able to measure the constructs involved.

Psychologists research things that are usually quite abstract: intelligence, attraction, aggression, mental health, and similarity, for example. We call these "constructs" or "psychological constructs". One of the first tasks that a researcher must face is the translation of these abstract, vague, squishy constructs into something specific, concrete, and ultimately, observable and therefore measurable. We refer to this as the process of operationalisation. There are no hard and fast rules about how to operationalise a construct as there are many different components to the decision process. These include what is possible given your area of study, what will be most convincing to the readers of your research, and what you can do given the resources (time, personnel) available.

In some cases, operationalisation is quite simple and there is no controversy or ambiguity. Concrete constructs such as gender, birth order, weight, and shoe size are easily and uncontroversially measured. But even so, they still need to be operationalised. But most of the constructs of interest to psychologists are not so clear cut. Suppose, for example, we wanted to examine the effectiveness of a new educational television program aimed at reducing childhood aggression. The plan is to show one group of children in a classroom the program and not show it to another group. To examine whether or not exposure to this program has an effect on childhood aggression, we need to measure how aggressive a child is. How would we go about measuring how aggressive a child is?

Fortunately, there are many different ways that this could be accomplished. For example, as shown in Figure 2.1, we could actually observe the childŐs behaviour in the classroom for a period following exposure to the program and count up the number of aggressive acts (eg., punching, teasing, name calling) we see in the two hours after the program. Or we could have the childŐs teacher or parents rate the childŐs aggressiveness on some scale (say 1 to 7) each day after being exposed or not to the program. The higher the rating, the higher is the aggressiveness of the child. We could have all the children in the classroom nominate the 5 "meanies" in the class and count the number of times each child is nominated. We could place each child in some artificial social situation where he or she is given an opportunity to strike or punch a doll and observe if the child does lash out in this situation ("test situation" in Figure 2.1). We could tally up the number of times the child is sent to the school headmaster or principal for abusing another child. So there are lots of ways of operationalising childhood aggressiveness. Now that we have come up with ways of measuring the construct, we can select the one or ones that seem best suited for our circumstances.

Figure 2.1 The operationalisation of constructs

Figure 2.1 The operationalisation of constructs

It is worth continuing with another example. Suppose we want to measure how depressed a person is. How might we do this? There are many ways. We could ask the person if they are depressed (yes or no) or to rate his or her depression on a 1 to 7 scale. We could ask the personŐs parents or friends if he or she could be characterised as depressed. We could have the person evaluated by a clinical psychologist to see if the person exhibits clinical signs of depression. We could have the person take a test such as the Beck Depression Inventory (a standardised self-report questionnaire designed to measure depression). We could ask the person how many events he or she recently experienced that we know often precede depression (such as the loss of a job, death of a friend, failing an exam). We could examine how much of a neurotransmitter called serotonin the person has (depressed people tend to have deficiencies in this neurotransmitter). We could evaluate glucose metabolism with a technique called Positron emission tomography (depressed people often donŐt metabolise glucose very actively). The important thing that you should see is the operational definitions turn the vague construct into something that can be measured and therefore quantified. We canŐt observe depression directly and therefore it canŐt be directly measured, but we can indirectly measure it if it is appropriately operationalised.

LetŐs try another example to make sure that the point is clear. Suppose you are studying the effects of praise on the attractiveness of a person to others. That is, you propose that people who have just been given positive information about themselves will be perceived as "attractive to others" through some mechanism. For this study, your participants take a test in some area important to them. Half of the participants are then told that they did very well on the test. Indeed, they are told that they scored in the top 5% of all people who have ever taken this test. The other half of the participants are not given any feedback about their performance. Following this, each participant is asked to engage in a 10-minute cooperative task with a stranger to the participant (the stranger is also a participant in the study!). You predict that participants who take the test and are then given positive feedback about themselves will be perceived as more attractive to the stranger.

Now, how might you measure "perceived attractiveness"? You could directly ask the stranger how attracted he or she was to the participant. You could ask this question quite vaguely and perhaps get a rating from 1 to 7. Or you could have the stranger rate the person on a variety of personality dimensions that vary in how well they capture the overriding psychological construct of "attractiveness to others". An "adjective checklist" such as "honest", "pleasant", "positive", "irritable", "gloomy", "unpleasant to be with" could be responded to on a scale from 1 to 7 where higher scores indicate more of this construct. Perceived attractiveness could be measured as the sum of the ratings on the "good" characteristics minus the sum of the ratings on the "bad" characteristics. In such a variety of adjectives we would also consider a number of other things such as an equal number of positive and negative descriptors.

You could actually measure the amount of time during the 10-minute interaction the stranger spends looking at the participant (we tend to look more at people we are attracted to). Or you could videotape the whole interaction and have a set of observers watch the tape and judge how attracted the stranger is to the participant.

We are going to compare one group of people who get positive feedback and one group that does not get positive feedback. The thing that we are manipulating here is "positive feedback". One group gets it (and is therefore referred to as the "Experimental group" and one group does not (and therefore referred to as the "Control group"). Notice that "positive feedback" is also a psychological construct and also needs to be operationalised. We have operationalised "positive feedback" here as giving participants in the experimental group the information that they scored in the top 5% of the people who have ever taken this test. The participantsŐ actual score is irrelevant. Each participant is given the same information about their performance and they are given the information in a way that is likely to ensure that the participant actually believes the information. It would be no good if some participants caught on that this was all a ruse. In this case, your manipulation would not have worked and the whole logic behind the study would collapse.

You might find it worthwhile to think up a number of operationalisations for the following constructs,: shyness, anger, happiness, and impatience. You will probably find that, first of all, almost anything can be operationalised if you try hard enough. Second, you will probably be surprised at how easy it is to come up with operational definitions of some constructs, while others are more difficult. Third, you will probably find that it is fun to do this. The translation of constructs into operational definitions is one place where creativity enters the scientific process.

Keep in mind as you work through this unit and others in your student career that one of the major difficulties students of psychology often have is losing sight of the fact that ultimately we are NOT interested in the specific operationalisations of the things we are studying. What we are interested in is the constructs the operationalisations represent. Because most constructs can be measured in lots of different ways, the research findings will mostly likely depend on the operationalisation used and how well the operationalisation "maps on to" the construct. Whether or not the operationalisation is a good representation of the construct will in many ways make or break a study, as described next.




© Copyright 2000 University of New England, Armidale, NSW, 2351. All rights reserved

UNE homepage Maintained by Dr Ian Price
Email: iprice@turing.une.edu.au