3 Types of studies
You have learnt how to ask a RQ. In this chapter, you will learn to:
- identify and describe the types of quantitative research studies.
- compare and contrast experimental and observational studies.
- describe and identify the directionality in observational studies.
- describe and identify true experimental and quasi-experimental studies.
- explain external and internal validity.
3.1 Introduction
Chapter 2 introduced four types of research questions: descriptive, relational, repeated-measures and correlational. This chapter discusses the types of research studies needed to answer these RQs, while Chaps. 4 to 10 discuss the details of designing these studies and collecting the data.
3.2 Three types of research designs
The RQ implies what data must be collected from the individuals in the study (the response and explanatory variables)... but the data can be collected in many different ways. Different types of studies are used to answer different types of RQs:
- descriptive studies (Sect. 3.3) answer descriptive RQs;
- observational studies (Sect. 3.4) answer RQs with an explanatory variable but no intervention; and
- experimental studies (Sect. 3.5) answer RQs with an explanatory variable, that have an intervention.
Observational and experimental studies are sometimes collectively called analytical studies.
3.3 Descriptive studies
Descriptive studies answer descriptive RQs.
Definition 3.1 (Descriptive study) Descriptive studies answer descriptive research questions.
Example 3.1 (Descriptive study) A study in Hong Kong determined the percentage of people wearing face-masks under different circumstances (L. Y. Lee et al. 2020). This is a descriptive study, where the population is 'residents of Hong Kong', and the outcome is (for example) 'the percentage who wear face masks when taking care of family members with fever'.
We do not explicitly discuss descriptive studies further, as the necessary ideas are present in the discussion of observational and experimental studies.
3.4 Observational studies
Observational studies are used for RQs with an explanatory variable but no intervention. They are commonly-used, and sometimes are the only type of research design possible.
Definition 3.2 (Observational study) Observational studies study relationships without an intervention.
Definition 3.3 (Condition) The conditions are the values of the explanatory variable that those in the observational study experience, but are not imposed by the researchers.
Example 3.2 (Observational study) Consider again this one-tailed, decision-making RQ (from Sect. 2.5):
Among Australian teenagers with a common cold, is the average duration of cold symptoms shorter for teens taking a daily dose of echinacea compared to teens taking no medication?
This RQ has a between-individuals comparison, so is a relational RQ. If the researchers do not impose the taking of echinacea (that is, the individuals make this decision themselves), the study is observational. The two conditions are 'taking echinacea', and 'not taking echinacea' (Fig. 3.1).
FIGURE 3.1: Observational studies with a relational RQ. The dashed lines indicate steps not under the control of the researchers.
Example 3.3 (Within-individuals relational study) D. A. Levitsky, Halbmaier, and Mrdjenovic (2004) recorded the weights of university students at the beginning university, and then after \(12\) weeks from the same students. The comparison is within individuals, and the study is a repeated measures (paired) study. Since the researchers do not impose anything on the students; there is no intervention (Fig. 3.2). The outcome is the average weight. The response variable is the weight of individuals. The within-individuals comparison is the number of weeks after university started (\(0\) and \(12\)).
FIGURE 3.2: Observational studies with a repeated-measures RQ. The dashed lines indicate steps not under the control of the researchers.
Example 3.4 (Correlational study) Poovaragavan et al. (2023) explored the relationship between time since death, and the concentration of sodium in synovial (knee) fluid. This is a correlational RQ as groups are not being compared, where the time since death is the explanatory variable, and the concentration of sodium in synovial fluid is the response variable. The researchers do not impose the time since death, so there is no intervention (Fig. 3.3).
FIGURE 3.3: Observational studies with a correlational RQ. The dashed lines indicate steps not under the control of the researchers.
3.5 Experimental studies
Experimental studies, or experiments, are used for RQs with an explanatory variable and an intervention, and are commonly-used. Well-designed experimental studies can establish a cause-and-effect relationship between the response and explanatory variables. However, using experimental studies is not always possible.
Definition 3.4 (Experiment) Experimental studies (or experiments) study relationships with an intervention.
Definition 3.5 (Treatments) The treatments are the values of the explanatory variable that the researchers impose upon the individuals in the experimental study.
In an experimental study, the unit of analysis (Def. 2.17) is the smallest collection of units of observations that can be randomly allocated to separate treatments.
Example 3.5 (Within-individuals experimental study) Consider this estimation RQ:
For obese men over \(60\), what is the average increase in heart rate after walking \(400\) m?
This RQ uses a within-individuals comparison (before and after walking \(400\) m) so is a repeated-measures (and paired) RQ. There is an intervention if researchers impose the \(400\) m walk on the subjects. The outcome is the average heart rate. The response variable is the heart rate for each individual man.
FIGURE 3.4: Experimental studies with a repeated-measures RQ. The dashed lines indicate steps not under the control of the researchers.
| Study type | Individuals allocated to receive the comparison? | Individuals allocated to treatments | Reference |
|---|---|---|---|
| Observational | No | No | Sect. 3.4 |
| True experiment | Yes | Yes | Sect. 3.5.1 |
| Quasi-experiment | No | Yes | Sect. 3.5.2 |
Example 3.6 (Correlational study) Xu et al. (2023) studied leaf-drip irrigation, exploring the relationship between the water pressure and flow rate. This is a correlational RQ, where the hydraulic pressure time is the explanatory variable, and the flow rate is the response variable. The researchers imposed nine different values for water pressure, so there is an intervention (Fig. 3.5).
FIGURE 3.5: Experimental studies with a correlational RQ. The dashed lines indicate steps not under the control of the researchers.
Between-individuals experimental studies can be either true experiments (Sect. 3.5.1 or quasi-experiments (Sect. 3.5.2); see Table 3.1.
3.5.1 True experimental studies
True experiments are commonly used to answer relational RQs. An example of a true experiment is a randomised controlled trial, often used in drug trials.
Definition 3.6 (True experiment) In a true experiment, the researchers:
- allocate treatments to groups of individuals (i.e., allocate the explanatory variable to the individuals), and
- determine who or what individuals are in those groups.
While the steps may not happen explicit, they can happen conceptually.
Example 3.7 (True experiment) The echinacea study (Sect. 2.11) could be designed as a true experiment. The researchers would allocate individuals to one of two groups, and then decide which group took echinacea and which group did not (Fig. 3.6).
These steps may happen implicitly: researchers may allocate each person at random to one of the two groups (echinacea; no echinacea). This is still a true experiment, since the researchers could decide to switch which group receives echinacea; ultimately, the decision is still made by the researchers.
FIGURE 3.6: True experimental studies: researchers allocate individuals to groups, and treatments to groups
3.5.2 Quasi-experimental studies
Quasi-experiments are similar to true experiments (i.e., answer relational RQs) where treatments are allocated to groups that already exist (e.g., may be naturally occurring).
Definition 3.7 (Quasi-experiment) In a quasi-experiment, the researchers:
- allocate treatments to groups of individuals (i.e., allocate the values of the explanatory variable to the individuals), but
- do not determine who or what individuals are in those groups.
Example 3.8 (Quasi-experiments) The echinacea study (Sect. 2.11) could be designed as a quasi-experiment. The researchers could find two existing groups of people (say, from Suburbs A and B), then decide to allocate people in Suburb A to take echinacea, and people in Suburb B to not take echinacea (Fig. 3.7).
FIGURE 3.7: Quasi-experimental studies: researchers do not allocate individuals to groups, but do allocate treatments to groups. The dashed lines indicate steps not under the control of the researchers.
Example 3.9 (Quasi-experiments) A researcher wants to examine the effect of an alcohol awareness program (based on M. MacDonald (2008)) on the average amount of alcohol consumed per student in a university Orientation Week. She runs the program at University A only, then compares the average amount of alcohol consumed per person at two universities (A and B).
This study is a quasi-experiment since the researcher did not (and can not) determine the groups: the students (not the researcher) would have chosen University A or University B for many reasons. However, the researcher did decide whether to allocate the program to University A or University B.
3.6 Comparing study types
In experimental studies, researchers create differences in the values of the explanatory variable through allocation, and then note the effect this has on the values of the response variable. In observational studies, researchers observe differences in the values of the explanatory variable, and observe the values of the response variable.
Importantly, only well-designed true experiments can show cause-and-effect. Nonetheless, well-designed observational and quasi-experimental studies can provide evidence to support cause-and-effect conclusions, especially when supported by other evidence. Although only true experimental studies can show cause-and-effect, true experimental studies are often not possible for ethical, financial, practical or logistical reasons.
The advantages and disadvantages of each study type are discussed later (Sect. 9.2), after these study types are discussed in greater detail in the following chapters.
Example 3.10 (Cause and effect) Many studies report that the bacteria in the gut of people on the autism spectrum is different than the bacteria in the gut of people not on the autism spectrum (Kang et al. (2019), Ho et al. (2020)), and suggest the bacteria may contribute whether a person is autistic. These studies were observational, so the suggestion of a cause-and-effect relationship may be inaccurate.
Other studies (Yap et al. 2021) suggest that people on the autism spectrum are more likely to be 'picky eaters', which contributes to the differences in gut bacteria.
The animation below compares observational, quasi-experimental and true experimental designs.
FIGURE 3.8: The three main research designs
3.7 Directionality
Analytical research studies (observational; experimental) can be classified by their directionality (Table 3.2):
- Forward direction (Sect. 3.7.1): The values of the explanatory variable are obtained, and the study determines what values of the response variable occur in the future. All experimental studies have a forward direction.
- Backward direction (Sect. 3.7.2): The values of the response variable are obtained, then the study determines what values of the explanatory variable occurred in the past.
- No direction (Sect. 3.7.3): The values of the response and explanatory variables are obtained at the same time.
Directionality is important for understanding cause-and-effect relationships. If the comparison occurs before the outcome is observed, a cause-and-effect relationship may be possible. That is, studies with a forward direction are more likely to provide evidence of causality.
| Type | Explanatory variable | Response variable |
|---|---|---|
| Forward direction | When study begins | Determined in the future |
| Backward direction | Determined from the past | When study begins |
| No direction | When study begins | When study begins |
Example 3.11 (Directionality) In South Australia in 1988--1989, \(25\) cases of legionella infections (an unusually high number) were investigated (O’Connor et al. 2007). All \(25\) cases were gardeners.
Researchers compared \(25\) people with legionella infections with \(75\) similar people without the infection. The recent use of potting mix was associated with an increase in the risk of contracting illness.
This study has a backward direction: people were identified with an infection, and then the researchers looked back at past activities.
Research studies are sometimes described as 'prospective' or 'retrospective', but these terms can be misleading (Ranganathan and Aggarwal 2018) and their use not recommended (Vandenbroucke et al. 2014).
Experimental studies always have a forward direction. Observational studies may have any directionality, and may be given different names accordingly.
3.7.1 Forward-directional studies
All experimental studies have a forward direction, and include randomised controlled trials (RCTs) and clinical trials.
Observational studies with a forward direction are often called cohort studies. Both experimental studies and cohort studies can be expensive and tricky: tracking a group of individuals (a cohort) into the future is not always easy, and the ability to track some individuals into the future may be lost (drop outs): plants or animals may die; people may move or decide to no longer participate, etc. Forward-directional observational studies:
- may add support to cause-and-effect conclusions, since the comparison occurs before the outcome (only well-designed experimental studies can establish cause-and-effect).
- can examine many different outcomes in one study, since the outcome(s) occur in the future.
- can be problematic for rare outcomes, as the outcome of interest may not appear (or may appear rarely) in the future.
Example 3.12 (Forward study) Chih et al. (2018) studied dogs and cats who had been recommended to receive intermittent nasogastric tube (NGT) aspiration for up to \(36\) hrs. Some pet owners did not give permission for NGT, while some did; thus, whether the animal received NGT was not determined by the researchers (the study is observational). The researchers then observed whether the animals developed hypochloremic metabolic alkalosis (HCMA) in the next \(36\) hrs.
Since the explanatory variable (whether NGT was used or not) was recorded at the start of the study, and the response variable (whether HCMA was observed or not) was determined within the following \(36\) hrs, this study has a forward direction.
3.7.2 Backward-directional studies
Observational studies with a backward direction are often called case-control studies. Researchers find individuals with specific values of the response variable (cases and controls), and determine values of the explanatory variable from the past. Case-control studies:
- only allow one outcome to be studied, since individuals are chosen to be in the study based on the value of the response variable of interest.
- are useful for rare outcomes: the researchers can purposely select large numbers with the rare outcome of interest.
- do not effectively eliminate other explanations for the relationship between the response and explanatory variables (called confounding; Def. 6.6).
- may suffer from selection bias (Sect. 5.11), as researchers try to locate individuals with a rare outcome.
- may suffer from recall bias (Sect. 10.2.2) when the individuals are people: accurately recalling the past can be unreliable.
Example 3.13 (Backwards study) Pamphlett (2012) examined patients with and without sporadic motor neurone disease (SMND), and asked about past exposure to metals.
The response variable (whether or not the respondent had SMND) is assessed when the study begins, and whether or not subjects had exposure to metals (explanatory variable) is determined from the past. This observational study has a backward direction.
3.7.3 Non-directional studies
Non-directional observational studies are called cross-sectional studies. Cross-sectional studies:
- are good for findings associations between variables (which may or may not be causation).
- are generally quicker and cheaper than other types of studies.
- are not useful for studying rare outcomes.
- do not effectively eliminate other explanations for the relationship between the response and explanatory variables (called confounding; Def. 6.6).
Example 3.14 (Non-directional study) J. Russell et al. (2014) asked older Australian their opinions of their own food security, and recorded their living arrangements. Individuals' responses to both both the response variable and explanatory variable were gathered at the same time. This observational study is non-directional.
3.8 The role of research design
Choosing the type of study is only one part of research design; many other decisions must be made also. The purpose of these research design decisions is to ensure researchers can focus on the relationship of interest (internal validity) in the population of interest (external validity).
Designing a study to maximise internal validity means:
- identifying what else might influence the values of the response variable, apart from the explanatory variable (Chap. 6); and
- designing the study to be effective (Chaps. 7 and 8).
In general, experimental studies have better internal validity than observational studies.
Designing a study to maximise external validity means:
- identifying who or what to study, since the whole population cannot be studied (Chap. 5); and
- determining how many individuals to study. (We need to learn more before we can answer this critical question in Chap. 30.)
The details of how the data will be collected (Chap. 10) and ethical issues (Chap. 4) must also be considered. Furthermore, the limitations of the study must be communicated (Chap. 9).
The following short (humourous) video demonstrates the importance of understanding the design!
3.9 Chapter summary
Three types of research studies are: Descriptive studies (for descriptive RQs), observational studies (for studying relationships without an intervention), and experimental (for studying relationships with an intervention).
Observational studies can be classified as having a forward direction (cohort studies), backward direction (case-control studies), or no direction (cross-sectional studies). Experimental studies always have a forward direction. Relational RQs with an intervention can be classified as true experiments or quasi-experiments. Cause-and-effect conclusions can only be made from well-designed true experiments.
Ideally studies should be designed to be internally and externally valid. In general, experimental studies have better internal validity than observational studies.
The following short videos may help explain some of these concepts:
3.10 Quick review questions
- A study (Fraboni et al. 2018) examined the 'red-light running behaviour of cyclists in Italy'.
This study is most likely to be:
- What is the difference between a true and a quasi-experiment?
- In a quasi-experiment, the researchers allocate treatments to groups that they cannot manipulate. True or false?
- True experiments generally have a higher internal validity than observational studies. True or false?
- Observational studies generally have a higher external validity than quasi-experimental studies. True or false?
3.11 Exercises
Answers to odd-numbered exercises are available in App. E.
Exercise 3.1 Consider this RQ (McLinn et al. 1994):
In children with acute otitis media, what is the difference in the average duration of symptoms when treated with cefuroxime compared to amoxicillin?
- Is the comparison a within- or between-individuals comparison?
- Is this RQ relational, repeated-measures or correlational?
- Is there likely an intervention?
- Is the RQ an estimation or decision-making RQ?
- Is the study observational or experimental? If observational, what is the direction? If experiment, is this a quasi-experiment or true experiment?
Exercise 3.2 Khair et al. (2015) studied the time needed for organic waste to turn into compost. For some batches of compost, earthworms were added. In other batches, earthworms were not added to the waste.
One RQs asked whether the composting times for waste with and without earthworms was the same or not.
- Is the comparison a within- or between-individuals comparison?
- Is this RQ relational, repeated-measures or correlational?
- Is there an intervention?
- Is the RQ an estimation or decision-making RQ?
- Is the study observational or experimental? If observational, what is the direction? If experiment, is this a quasi-experiment or true experiment?
Exercise 3.3 Gonzalez-Fonteboa and Martinez-Abella (2007) studied recycled concrete beams. Beams were divided into three groups, different loads were then applied to each group, then the shear strength needed to fracture the beams was measured. Is this a quasi-experiment or a true experiment? Explain.
Exercise 3.4 A research study compared the use of two different education programs to reduce the percentage of patients experiencing ventilator-associated pneumonia (VAP). Paramedics from two cities were chosen to participate. Paramedics in City A were chosen to receive Program 1, and paramedics in the other city to receive Program 2.
- Is this RQ relational, repeated-measures or correlational?
- Is the comparison a within- or between-individuals comparison?
- Is there likely an intervention?
- Is the study observational or experimental? If observational, what is the direction? If experiment, is this a quasi-experiment or true experiment?
Exercise 3.5 Manzano et al. (2013) compared 'the effectiveness of alternating pressure air mattresses vs. overlays, to prevent pressure ulcers' (p. 2099). Patients were provided with alternating pressure air overlays (in 2001) or alternating pressure air mattresses (in 2006). The number of pressure ulcers were recorded.
This study is experimental, because the researchers provided the mattresses. Is this a true experiment or quasi-experiment? Explain.
Exercise 3.6 Consider this initial RQ (based on Friedmann and Thomas (1985)), that needs refining: 'Are people with pets healthier?'
- Briefly describe useful and practical definitions for P, O and C.
- Briefly describe an experimental study to answer the RQ.
- Briefly describe an observational study to answer the RQ.
Exercise 3.7 Consider this journal article extract (Sacks et al. (2009), p. 859):
We randomly assigned \(811\) overweight adults to one of four diets [...] The diets consisted of similar foods [...] The primary outcome was the change in body weight after \(2\) years in [...] comparisons of low fat versus high fat and average protein versus high protein and in the comparison of highest and lowest carbohydrate content.
- What is the between-individuals comparison?
- What is the within-individuals comparison?
- Is this study observational or experimental? Why?
- Is this study a quasi-experiment or a true experiment? Why?
- What are the units of analysis?
- What are the units of observation?
- What is the response variable?
- What is the explanatory variable?