Like a true experiment, a Quasi-Experimental Design aims to establish a cause-and-effect relationship between an independent variable and a dependent variable. However, unlike a true experiment, a quasi-experiment is not based on random assignment. Instead, subjects are assigned to groups based on non-random criteria.
Quasi-experimental design is a useful tool in situations where real experiments cannot be used for ethical or practical reasons.
What is Quasi-Experimental Research?
The prefix quasi means “similar”. Therefore, quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or condition orders.
As the independent variable is manipulated before measuring the dependent variable, quasi-experimental research eliminates the problem of directionality. But since participants are not randomly assigned—making it likely that there are other differences between the conditions—quasi-experimental research does not eliminate the problem of confounding variables. Therefore, in terms of internal validity, quasi-experiments are usually halfway between correlational studies and true experiments.
Quasi-experiments are usually performed in field environments where random assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment, perhaps a type of psychotherapy or an educational intervention. There are many different types of quasi-experiments, but here we will discuss only some of the most common ones.
Example of a true experiment versus a quasi-experiment
Suppose you are interested in the impact of a new psychological therapy on patients with depression.
Example: True Experimental Design
To conduct a true experiment, half of the patients in a mental health clinic are randomly assigned to receive the new treatment. The other half, the control group, receives the standard treatment for depression.
Every few months, patients fill out a sheet describing their symptoms to see if the new treatment produces significantly better (or worse) effects than the standard one.
However, for ethical reasons, mental health clinic directors may not give you permission to randomly assign your patients to treatments. In this case, you cannot perform a real experiment.
Instead, you can use a quasi-experimental design.
Example: Quasi-Experimental Design
He finds that some of the clinic’s psychotherapists have decided to try the new therapy, while others treating similar patients have chosen to stick to normal protocol.
You can use these pre-existing groups to study the evolution of symptoms of patients treated with the new therapy versus those receiving standard treatment.
Although the groups were not randomly assigned, if any systematic differences between them are properly taken into account, one can be reasonably certain that any differences should arise from treatment and not from other confounding variables.
Types of Quasi-Experimental Designs
There are many types of quasi-experimental designs. Here we explain three of the most common types: the design of non-equivalent groups, discontinuous regression, and natural experiments.
Design of non-equivalent groups
Recall that when the participants of an experiment between subjects are randomly assigned to the conditions, the resulting groups are likely to be quite similar. In fact, researchers consider them equivalent. However, when participants are not randomly assigned to the conditions, the resulting groups are likely to be different in some respects. For this reason, researchers consider them non-equivalent. A non-equivalent group design, therefore, is a design between subjects in which participants have not been randomly assigned to the conditions.
Imagine, for example, a researcher who wants to evaluate a new method of teaching fractions to third-graders. One way would be to conduct a study with a treatment group consisting of a class of third-graders and a control group made up of another class of third-graders. This design would be a non-equivalent group design because students are not randomly assigned to classes by the researcher, meaning there could be important differences between them.
For example, parents of higher-performing or more motivated students may have requested more often that their children be assigned to Ms. Williams’ class. Or the principal could have assigned the “troublemakers” to Mr. Jones because he’s more disciplinary. Of course, teacher styles, and even classroom environments, could be very different and lead to different levels of performance or motivation among students. If at the end of the study there was a difference in knowledge of the fractions of the two classes, it could have been caused by the difference between teaching methods, but it could have been caused by either of these confounding variables.
Measures to Consider in the Design of Non-Equivalent Groups
Of course, researchers using a non-equivalent group design can take steps to ensure that their groups are as similar as possible. In the present example, the researcher might try to select two classes in the same school, where students in the two classes have similar results on a standardized math test and the teachers are of the same sex, close in age, and have similar teaching styles. Taking these steps would increase the internal validity of the study because it would eliminate some of the most important confounding variables. But without a true random assignment of students to the conditions, there remains the possibility of other important confounding variables that the researcher could not control.
In the design of non-equivalent groups, the researcher chooses existing groups that appear similar, but in which only one of the groups experiences treatment.
In a real randomized experiment, the control and treatment groups are considered equivalent in all respects except treatment. But in a quasi-experiment in which the groups are not random, they may differ in other respects: they are non-equivalent groups.
When using this type of design, researchers try to take into account the confounding variables by controlling them in their analysis or choosing groups that are as similar as possible.
This is the most common type of quasi-experimental design.
Example: Designing Non-Equivalent Groups
It is hypothesized that a new after-school programme will lead to better grades. Two similar groups of children attending different schools are chosen, one of whom applies the new programme and the other does not.
By comparing children who attend the program with those who don’t, you can find out if it has an impact on grades.
Discontinuity of regression
Many potential treatments that researchers want to study are designed around an essentially arbitrary threshold, in which those above the threshold receive the treatment and those below do not.
Near this threshold, the differences between the two groups are usually so minimal that they are almost non-existent. Therefore, researchers can use individuals who are just below the threshold as a control group and those who are just above as a treatment group.
Example: Discontinuity of regression
Some U.S. high schools are reserved for high-achieving students, who must pass a certain score on an exam in order to attend. Most likely, those who pass this test will systematically differentiate themselves from those who do not.
However, since the exact limit score is arbitrary, students who are close to the threshold – those who barely pass the exam and those who fail it by a very small margin – are usually very similar, and the small differences in their scores are mainly due to chance. Therefore, it can be concluded that any difference in outcomes must come from the school they attended.
To check the impact of attending a selective school, the long-term results of these two groups of students (those who barely passed and those who barely failed) can be studied.
Natural experiments
In both laboratory and field experiments, researchers typically control which group subjects are assigned to. In a natural experiment, an external event or situation (“nature”) results in the random or random assignment of subjects to the treatment group.
Although some use random assignments, natural experiments are not considered true experiments because they are observational in nature.
Although researchers have no control over the independent variable, they can exploit this event a posteriori to study the effect of treatment.
Example: Natural experiment
The Oregon Health Study is one of the most famous natural experiments. In 2008, the state of Oregon decided to expand enrollment in Medicaid, America’s low-income public health insurance program, to more low-income adults.
However, since they could not afford to cover all the people they considered eligible for the program, they instead allocated places in the program based on a random draw.
The researchers were able to study the impact of the program using people enrolled as a randomly assigned treatment group, and other people who met the requirements but had not won the lottery as a control group.
Pre-test-post-test design
In a pretest-postest design, the dependent variable is measured once before applying the treatment and once after applying it. Imagine, for example, a researcher interested in the effectiveness of an anti-drug education program on the attitudes of primary school students towards illegal drugs. The researcher could measure the attitudes of students in a particular primary school for a week, apply the anti-drug program during the following week, and finally re-measure their attitudes the following week.
The pretest-posttest design closely resembles an intra-subject experiment in which each participant is evaluated first under the control condition and then under the treatment condition. However, it differs from an intra-subject experiment in that the order of conditions is not counterbalanced because it is not normally possible for a participant to be evaluated first in the treatment condition and then in an “untreated” control condition.
Measures to Consider in Pretest Design – Posttest
If the average score after the test is better than the average score before the test, then it makes sense to conclude that treatment could be responsible for the improvement. Unfortunately, this often cannot be concluded with a high degree of certainty because there may be other explanations for why post-test scores are better. A category of alternative explanations is called history.
Other things may have happened between the pre-test and the subsequent test. Maybe an anti-drug show was aired on TV and many of the students watched it, or maybe a celebrity died of a drug overdose and many of the students found out about it. Another category of alternative explanations is called maturation. Participants may have switched between the pre-test and the subsequent test in a way they were going to do anyway because they are growing and learning. If it is a year-long program, participants could become less impulsive or reason better and this could be responsible for the change.
Another alternative explanation for a change in the dependent variable in a pretest-postest design is regression to the mean. This refers to the statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion. For example, a bowler with a long-term average of 150 who suddenly bowls with 220, will almost certainly get a lower score in the next match. Your score will “go back” to your average score of 150.
Regression to the mean
Regression to the mean can be a problem when participants are selected for a subsequent study because of their extreme scores. Imagine, for example, that only students who scored especially low on a fraction exam receive a special training program and are then re-examined. Regression to the mean virtually guarantees that your scores will be higher even if the training program has no effect. A closely related concept – and extremely important in psychological research – is spontaneous remission. This is the tendency of many medical and psychological problems to improve over time without any treatment.
The common cold is a good example. If the severity of the symptoms of 100 people suffering from a common cold were measured today, given a bowl of chicken soup every day, and the severity of the symptoms were remeasure within a week, they would probably get much better. However, this does not mean that chicken soup is responsible for the improvement, as they would have improved a lot without any treatment. The same goes for many psychological problems.
A group of severely depressed people today are likely to be less depressed on average within 6 months. Reviewing the results of several studies on treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in waiting list control conditions improved an average of 10 to 15% before receiving any treatment (Posternak & Miller, 2001)[2]. Therefore, in general, you have to be very cautious when inferring the causality of pretest-postest designs.
Interrupted time series design
A variant of the pretest-postest design is the interrupted time series design. A time series is a set of measurements made at intervals over a period of time. For example, a manufacturing company can measure the productivity of its workers every week for a year. In an interrupted time series design, a time series like this is “interrupted” by a treatment.
In a classic example, the treatment was the reduction of work shifts in a factory from 10 to 8 hours. Since productivity increased quite rapidly after the reduction of work shifts, and because it remained high for many months afterwards, the researcher concluded that the reduction in shifts caused the increase in productivity. Note that the design of interrupted time series is like a pretest-postest design in that it includes measurements of the dependent variable both before and after treatment. However, it differs from the pretest-postest design in that it includes multiple pretest and postest measurements.
When to use a Quasi Experimental design
Although true experiments have greater internal validity, you can choose to use a quasi-experimental design for ethical or practical reasons.
Ethics
Sometimes it would be unethical to provide or retain a treatment randomly, so a true experiment is not feasible. In this case, a quasi-experiment may allow you to study the same causal relationship without the ethical problems.
The Oregon Health Study is a good example. It would be unethical to randomly provide health insurance to some people, but purposely prevent others from receiving it solely for research purposes.
However, since the Oregon government faced financial constraints and decided to provide health insurance through a lottery, studying this fact a posteriori is a much more ethical approach to studying the same problem.
Practice
A true experimental design may be unfeasible to apply or simply too expensive, especially for researchers who do not have access to large funding streams.
At other times, recruiting and properly designing an experimental intervention for an adequate number of subjects requires too much work to justify a true experiment.
In either case, quasi-experimental designs allow the issue to be studied by taking advantage of data that has previously been paid for or collected by others (often the government).
Advantages and disadvantages
Quasi-experimental designs have several advantages and disadvantages compared to other types of studies.
Greater external validity than most true experiments. They often involve real-world interventions rather than artificial laboratory environments.
Greater internal validity than other types of non-experimental research. They allow better control of confounding variables than other types of studies.
Less internal validity than real experiments. Without randomization, it can be difficult to verify that all confounding variables have been taken into account.
The use of retrospective data that has already been collected for other purposes may be inaccurate, incomplete or difficult to access.
Our specialists wait for you to contact them through the quote form or direct chat. We also have confidential communication channels such as WhatsApp and Messenger. And if you want to be aware of our innovative services and the different advantages of hiring us, follow us on Facebook, Instagram or Twitter.
If this article was to your liking, do not forget to share it on your social networks.
You may also be interested in: The Universe in Statistics
Bibliographic References
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.
Quasi-Experimental Design. Photo: Unsplash. Credits: Brooke Cagle
