The Experimental Research Strategy
Chapter 7: The Experimental Research Strategy
The goal of the experimental research strategy (p. 196)
In chapter 2 we noted that a research strategy is the general approach used to evaluate a research hypothesis
The goal of the Experimental Research Strategy is to demonstrate cause and effect relationships between variables
In an experiment we accomplish this by showing that changes in one variable are directly responsible for changes in another variable.
In order to do this every research study includes four basic procedures:
- Manipulation
- Measurement
- Comparison
- Control
The varialbe that is manipulated is called the independent variable
A treatment condition is characterized by having one specific value of the independent variable
The dependent variable is observed and measured to determine if it changes at different levels of the independent variable
Extraneous variables include all variables in a study other than the independent and dependent variables
With the above terminology in mind, we can say that in an experiment the investigator:
- Manipulates the independent varialbe
- Measures the dependent variable in each treatment condition
- Compares the value of the dependent variable at each level of treatment
- Controls (as much a spossible) extraneous variables
Manipulation (p. 202)
One of the more distinctive features of an experiment, as opposed to other research strategies, is that the investigator manipulates one of the variables being studied.
The investigator first decides what values or levels of the independent variable to study, and creates treatmetn conditions corresponding to those values.
For example,to study the effects of a medication on behavior the treatment conditions might be: no medication, a low dose, a high dose. To study the effect of sleep deprivation on memory, the levels of sleep deprivation might be: no sleep for 24 hours (severe deprivation), 4 hours of sleep (moderate deprivation), 8 hours of sleep (no deprivation)
In other research strategies, such as the correlational strategy (discussed in a later module) the investogator measures all variables, but does not manipulate them.
The major advantage of manipulating a variable is that it allows an investigator to determine the direction of a relationship.
In correlational reseach we might note that two variables are related, and we cannot determine which variable is the casue and which is the effect.
This is know as: The directionality problem (p. 203)
The Directionality Problem: An example
Suppose I wanted to determine if playing violent video games cause violent behavior.
I might ask several teens to record how much time they spend playing violent video games during a one week period, and then asked parents and teachers to rate those teens with respect to various aggressive behaviors during that same one week period.
Now suppose I find that those who spend more time playing violent video games engage in more violent behavior...what can I conclude?
Does playing violent video games cause aggressive behavior?
(image of violent video game) (image of two boys fighting)
Or is it the other way around? That is, do people who are more aggressive choose to play more violent video games?
(image of two boys fighting) (image of violent video game)
In the correlational study described above, where neither variable is manipulated, causality cannot be determined.
In order to determine causaility I would have to do an experiment.
For example, I might find a group of teens who do not play video games, and have parents and teachers rate aggressive behavior for one week. And then I might manipulate exposure to violent video games by randomly assigning half of the teens play non-violent video games for two hours a day for one week, and assigning the other half to play violent video games for two hours a day for a week, and then having parents and teachers again rate aggressive behavior following exposure to video games.
By manipulating exposure to violent video games, if those who play violent video games become more aggressive than the group who play non-violent video games, I could be confident that the video games caused the increase in aggressive behavior, rather than the other way around.
By the way, the relationship between violent video games and violent behavior is a controversial topic. Here is a link to a very recent (February 13, 2013) New York Times article on the topic.
The writers must have done their research methods homework, as they correctly point to limited external validity as a major reason that it is difficult to draw clear conclusions from the research on video games and violent behavior...
The third variable problem (p. 204)
Manipulating variables also helps avoid the thrid variable problem.
In the real world, variables don't change in isolation, that is, many things can change at once.
In the laboratory we can manipulate the level of a single variable, and then we can be confident that any change in the dependent variable is due to changes in the independent vbariable rather than to chagnes in some third variable.
For example, suppose I notice that there is a relationship between cancer and driving luxury cars.
Specifically, people who drive luxury cars are more likely to get cancer in the next five years than people who don’t drive luxury cars.
(image of luxury car) (image of cancer patient)
Does this mean that driving a luxury car causes cancer?
NO – and this silly example does illustrate the ‘third variable problem’...
You have probably heard the saying that "correlation does not equal causation" - this saying is at the heart of the third variable problem.
Two variables can be correlated when both are caused by a third variable. Cancer and driving luxury cars are correlated. However, the relationship between them is not causal. One does not cause the other.
However, the probability of owning a luxury car AND the probability of getting cancer have a similar relationship to a third variable: namely AGE
(image of elderly man driving a car)
Older people are more likely to own luxury cars because they have more wealth, on average, than do younger adults.
And, the risk of cancer increases with age, so older people are more likely to get cancer than are younger adults.
While it might appear that luxury car ownership and cancer are causally related, in fact both are related to a third variable: age.
I could manipualte age, e.g., compare groups of older persons and younger persons and show that getting older is related to increased likelihood of getting cancer, or owning a luxury car.
Or, I could do a really silly experiment, and manipualte luxury car ownership...
that is, I might randomly assign two groups of people to recieve either an inexpensive car or a luxury car, and see how many develop cancer in the next five years, and I would undoubtedly find that owning a luxury car does not cause cancer!
Control
So far you have learned that in an experiment:
- We manipulate the independent variable.
- We measure the dependent variable at different levels of the independent variable
- And we then compare the measurements of the dependent variable in one or more treatment groups
What about all the other variables that might affect the dependent variable?
We use various strategies to CONTROL them so we can be confident that any change in the dependent variable is due to the level of the independent variable rather than to the effect of other varialbes (p. 205).
Dealing with extraneous variables (p. 207)
All variables other than the independent and dependent variable are extraneous variables.
An extraneous variable becomes a problem - a confounding variable - when it influences the dependent variable AND varies systematically with the independet variable (p. 209)
Make sure you understand what a confounding variable is!
In chapter six different categories of extraneous variables were identified and considered:
- Environmental variables
- Individual differences
- Time related variables
We deal with extraneous variables by using various techniques to control them
Control by Holding a Variable Constant (p. 209)
We hold a variable constant when we make sure it is equal for all particpants.
Environmental conditions: we can make sure the experiment is in the same room, that the level of noise or other distractions is constant, that the same researcher is present at all times, etc.
Individual differences: we make sure all the aprticipatns are the same age, or the same gender, or the same ethnicity, or have had the same amount of sleep the night before, etc.
Time related variables: we make sure the experiment takes palce at the same time of day, and that the duration fo teh experiment is the same for both groups, etc.
Advantages
Holding a variable constant eliminates it's potential to be a confound: if all the participants are women, then gender won't be a confound, if every participant completes the experiment in the same room at the same time of day then the room and time of day won't be confounds.
Disadvantages
There are two major disadvantages to holding a variable constant
1. it reduces generalizability/external validity
if all the participants are girls, gneder is not a confound, but we can't generalize the resutls to boys!
2. It is often impractical
if we want all partcipants to have the same IQ, then we have to measure IQ and it will take a long time to find a large group fo people with the exact same IQ
AND, we won't be able to generalize the results to poeple with other IQ's
So, why bother if it is impractical and limits generalizability? It might be a good start...for example, if we find that our experimental treatment works great in six year old boys, then our next experiment might be to study the same treatment in six year old girls, or 8 to 10 year old boys.
Control by Matching Across Samples (p. 210)
We can also exert control over extraneous variables by matching them across treatment conditions.
Supoose we must use two different rooms, or times of day, or genders...we can't hodl them constant, but we might be able to match them.
If one treatment group participated in one room and the other in another room, then room woudl be an environmental confound.
If one treatment group was all boys and the other all girls, then sex would be a confound.
If we couldn't control these variables, we might try to match the groups, so that an equal proportion in each group took place in each room, and that an equal number of boys and girls were in each group, and that an equal number of each treatment condition was in the morning versus the afternoon.
Advantages of Matching
The groups are equal with respect to the matched variables, so they can be eliminated as confounds.
Disadvantages
Simialr to holding variables constant, matching can be impractical
It is easy to do with one or a few variables that are easy to measure (gender, age, room) and gets harder when they are harder to measure or there are more of them, for eaxample, imagine tryign to match on age, gender, socioeconomic status, and IQ...
If one participant in one condition was a 67 year old, middle class male, with an IQ of 102, then you would have to find another participant just like him for the other group!
Control by Randomization (p. 211)
Randomization is the most common method of controlling extraneous variables.
You should recall that sampling procedures were discussed in chapter 5. When we have a procedure fro randomizing group assignment, AND a large enough sample, we shoudl ahve a fairly (but not perfectly) matched group. That is, if we recruit 100 participants and randomly assign them to one of two groups, we shoudl be reasonably confident that each group will have about - but not exactly - the same proportion of men and women, of about the same average age, and about the same average IQ.
If the procedure is random (see p. 211 for definition of a random procedure), and our sample size is not too small, then we can be reasonably confident that changes in the dependent variable are de to manipulation fo the independent variable rather than to differences in extraneous variables.
Can you see why small sample sizes are a problem???
The major advantage of randomization is that it is usually much simpler than matching and holding variables constant.
The major disadvantage is that randomization does not absolutely guarantee that variables are well-matched across groups.
If a researcher has good reason to think that a particular variable migth affect the dependent variable, then she should hold that variable constant or use a matching procedure with respect to that variable, and randomize with respec tot remaining variables.
Control Groups (p. 214)
Control groups are used when an investigator wants to investigate one treatment instead of comparing two treatments.
Common control group types include no treatment control groups, and placebo control groups,
We call a group in an experiment a control group when participants in the group recieve no active treatment.
A group receiving any treatment level above zero level is called an experimental group.
We usually think of a 'treatment' as a medication or therapy. However, technically speaking, any variable we manipulate in an experiment can be regarded as a 'treatment'. For example:
Control Groups: no therapy, no breakfast, no exercise, no television, no medication.
Corresponding Treatment Groups: attend therapy, eat breakfast, particpate in exercise program, watch television, take medication
Placebo Control Groups (p. 215)
Your textbook describes a placebo as an "inert or innocuous medication, a fake medical treatment. Another way to think about it is that it is a fake treatment or a non-treatment that looks like a real treatment.
In a basic control condition we tell particpants 'you get no tretament' in a placebo control condition we tell them 'you will be getting one of two treatments' and give them either the placebo or the experimental treatment.
We use placebo control groups in research to control for the placebo effect
The placebo effect
The placebo effect is not very well understood. A person recieving a placebo medication or treatment will feel better even though the treatment is inert, that is, it has no effect on the body. It is believed to be a psychosomatic or psychological response to expectations about treatment.
A person thinks they have taken headache medication, and they report decreased pain
A person beleives they have drunken alcohol, and they feel less inhibited
A person believes they have particpated in treatment for anxiety, and they feel more relaxed
Placebos in research
While some docotors admit to prescribing placebos tho their patients with hypochonriasis, placebos are more commonly used in research rather than practice.
As an aside, why might it be ethically problmeatic for a docotr to prescribe a placebo? Why do doctors do it in spite of ethcial concerns?
Answer - if they are giving a patient an inert treatment and not telling them, then the patient is not able to provide consent for treatment and it can undermine Dr./patient trust
. some doctors feel that the benefits outweight the risks when the patient may have a psychosomatic illness or hypochondriasis.
Placebo controls are used in research because sometimes a placebo will produce a clinically significant effect, that is, a person receiving a placebo will report more improvement than a person recieving no treatment.
Even if an experimental treatment is completley ineffective, some participants will report 'feeling better' due to a placebo effect.
A placebo control is amore useful control than a "no-treatment control", because we can determine whether the experiemtnal treatment has any effect above and beyond a placebo effect.
Many studies use three comparison groups: a no treatment control group, a placebo control group, and a treatment group
A few points about control groups...
- Not every experiment uses a control group.
- Control groups can be thought of as 'level zero' of the independent variable
- Placebo control groups only control for placebo effects; they do not control for extraneous variables.
Manipulation Checks (p. 217)
A manipulation check is a measure administered to research particpants in order to find out whether or how they perceived the manipulation fo the independent variable.
For example, if a researcher wanted to study the effect of humor on learning and had particpants read funny stories or boring stories before taking a memory test, then a manipulation check might ask the particpant to "please rate how funny you found each story."
The reason for the maipulation check is for the researchers to make sure particpants perceived the independent variable. For instance, in the humor example if non one in the 'funny story' condition rated the stories as humorous, then a null hypothesis result might mean that the stories were not funny rather than that humor has no effect on learning.
Make sure you understnad the defintion and purpose of a maipulation check!
Other examples:
- In a study on the effect of mood on task performance a researchers plays sad or happy music to manipulate partcipants' mood. A manipulation check might have the particpants rate their mood before and after the mood induction to make sure the music had the intended effect.
- In a study on the effects of alcohol on behavior with an alcohol and placebo control group a mainipualtionc heck might ask particpatns to rate how intoxicated they feel
- In a study where deception is used a researcher might ask participants "what do you think the purposeof the experiemnt was?" or "do you suspect that youw ere being decieved?"
Four situations where it is especially important to use a manipualtion check:
- The researcher is trying to change the particpant rathe rthan the environment. It is easy to know if you have changed the temperature of the room, or the color of the lighting, or the amount of alcohol in a beverage. It is less easy to be certain tha tyou ahve made a particpant feel embarrassed, or happy, or more relaxed, or bored, or frustrated.
- The manipulation is subtle. If the manipulation to the independent variable is subtle, such as a minor wording change in instructions, a younger Vs. older person providing instructions, instructions being provided in a room with white walls versus a room with pink walls, the researcher will want to determine if particpatns even noticed the manipulation.
- The study is a simulation study. See next section for more on simulations...and, when a researcher uses a simulation a manipulation check can determine how the particpants responded to the simulation. For instance, if the simulation was a driving simulation a researcher might ask particpants how similar hitting a tennis all tossed by a machine felt as compared to returning a serve hit by another tennis player.
- A placebo control is used. When a placebo control is used the researcher will almost always ask if the person believes they were receiving the placebo versus a non-inert treatment and/or how they believe the treatment - whether a placebo or active treatment - affected them.
Simulations and Field Studies (p. 218)
The major purpose of a simulation or field study is to increase external validity.
One problem with laboratory studies is that the laboratory environment itself threatens external validity. That is, we can't be sure if results obtained in the lab are the same as results we would obtain "in the real world."
Is receiving therapy in a lab the same as receiving therapy in a hospital? Is taking a test in a lab the same as taking a test in a classroom? Is responding to an emergency in a lab setting the same as responding to an emergency in the real world? Is drunken behavior in a lab the same as drunken behavior outside of a lab? Do children play the same in a lab as on a playground?
We can't be sure that results of a lab study will apply outside the lab. Simulation and field studies are two strategies for making the experimetnal setting more like 'the real world."
Simulation
In a simulation the researcher attempts to make the lab environemnt resemble 'the natural world.' She creates condtions in the lab that duplicate the look and feel of the natural environment where the behavior of interest usually occurs.
The infamous Stanford prison study is described in yoru textbook as an example of a study that had both high mundane realism and high experiemtnal realism.
(image of a 'prisoner' and 'guard' from Stanford prison experiment)
This study had high 'mundane realism' in that aspects of the prison constructed in the lab looked like the real thing.
It had high experimental realism in that the psychological effects of being in the experimental prison were much like those experienced by actual prisoners and prison guards. In fact, Zimbardo's simulation was so good that the experiment was suspended early because the partcipant prisoner's were becoming depressed and terrorized and the particpant guards were becoming brutal and hostile. The details of the study are described on p. 219 in yoru textbook.
(image of a car simulator) (image of a playground slide indoors)
Above are two examples of simulations with high mundane realism and low environmental realism. You probably would not 'drive' the car simulator the same as you woudl drive an actual car, and chilren probably would not play on the slide the same a sthey would play on a playground.
Q: how coudl you increase the 'experimental realism' of the car simulator?
A: One thing you might do is add audio and video with sounds a images of other cars an drivers.
Vidoe games are simulations, and many 'improvements' are aimed at increasing how they feel, that is, to affect the psychological experience of the player.
Field Studies
Field studies and simulations both have the purpose of increasing external validity and they do so by almost opposite means. If simulations are a way to bring the real world into the lab, then field studies can be thought of as getting the experiment of the lab and into the real world!
Suppose a researcher wants to study consumer behavior. In a structly lab-based study she might ask particpants questions about their shopping behavior, and In a simulation she might create a fake grocery, drug store, or botique and observe particpants' behavior.
In a field study she would conduct the experiment at an actual store.
Note that doing so does not guarentee theat consumenrs will beahve exactly as they would in the real world; a shpeer might behave differently while being follwed by a researcher than he would while on his own! However, particpants' behavior might be more simular to their real world, non-experimental beahvior in a field study than in a lab.
Advantages and Disadvantages of simulations and field studies (p. 221)
The advantage is improved external validity. The more life-like the experimental setting the more we can be confident tha tpartipants' behavior in the experiment will generalize to the natural environment.
The main advantage of a field study is that the researcher loses control over the environment. We can't predict what might happen while 'out in the field'. In fact, this lack of control is the reason we do research in labs in the first place!
Simulations afford the researcher with more control over the environment, however their success depends on the willingness of partipants to accept or buy into the simulation.