7.4 Other design considerations

Many issues must be kept in mind when designing studies, such as selecting the sample and managing confounding. In this section, some specific design considerations are discussed:

Not every design consideration will be relevant to every study.

7.4.1 Carry-over effect

In the Himalaya study, what if patients spent two weeks on the Himalaya 292 diet, then the next two weeks on the refined cereal diet?

Potentially, the influence of the first diet could still be impacting the subjects’ faecal weight for a little while after stopping the first diet. This is called the carryover effect.

Definition 7.2 (Carryover effect) The carry-over effect is when the influence of one treatment carries over into the influence of the next treatment.

The impact of the carryover effect can be minimized by using:

  • A washout period or similar; for example, after finishing one diet, the participants spend four weeks on their usual (before study) diet, and then revert to the second diet being used.
  • Random allocation: if possible, the order in which the treatments (i.e., the diets) are used should be randomly allocated. That is, some participants start by spending four weeks on the Himalaya 292 diet, then (after a washout period) four weeks on the refined cereal diet; meanwhile, other participants start by spending four weeks on the refined cereal diet, then (after a washout period) four weeks on the Himalaya 292 diet.

Example 3.7 (Carry-over effect) In the Himalaya 292 study, the authors report:

Subjects were allocated randomly to […] dietary treatments according to a cross-over study design with each intervention phase lasting 4 weeks. There was no washout period between phases.

Bird et al. (2008), p. 1033

That is, subjects were randomly allocated to a diet: some subjects began the study on the Himalaya 292 diet while others started on the refined cereal diet, No washout period was used; however, since the response variable was recorded after four weeks on the diets, no washout period was necessary.
Example 7.3 (Washout periods) An engineering study (Miller and Boyle 2019) examined drivers’ exposure to lane-keeping system on their driving performance. Subjects were exposed to a driving simulation that used a lane-keeping system, and then to a driving simulation without using a lane-keeping system. The researchers found that there was a carryover effect when drivers moved from a simulation with a lane-keeping system to one without a lane-keeping system.
Using a 'washout' period to minimize the carry-over effect

FIGURE 7.5: Using a ‘washout’ period to minimize the carry-over effect

7.4.2 Hawthorne effect

What if the patients in the Himalaya 292 study were being watched (or waited for) while defecating?

People often behave differently (either positively or negatively) if they know (or think) they are in a study or are being watched. This is called the Hawthorne effect (McCarney et al. 2007).

Definition 7.3 (Hawthorne effect) The Hawthorne effect is the tendency of people (or animals, or…) to behave differently if they know (or think) they are being observed.

Example 3.8 (Hawthorne effect) People are more health-conscious if they know they will be followed up on a regular basis. For example, a study aiming to increase fruit and vegetable intake in young adults (Clark et al. 2019) noted that

The changes that did occur could be explained by the Hawthorne effect […] the intervention […] can inherently cause participants to change behavior because they know they are being observed…

Clark et al. (2019)

The impact of the Hawthorne effect can be minimized by using:

  • Good planning: Sometimes, efforts can be made to reduce the chance that individuals know they are being observed. Often, however, this is not possible when studying people (as ethics usually requires individuals’ informed consent).
  • Blinding of participants: Blinding individuals may be possible; that is, the individuals do not know which treatment they are receiving. Note, however, that this is only a partial solution: the participants will almost certainly still know that they are in a study (especially when the individuals are people, as ethics usually requires individuals to be informed).

Example 7.4 (Hawthorne effect) In the Himalaya 292 study, the authors report:

The study was explained fully to the subjects, both verbally and in writing, and each gave their written, informed consent before participating.

Bird et al. (2008), p. 1033

That is, the subjects knew they were in a study. As is usual, this was an ethics requirement (in this case, from the Ethics Committee of the CSIRO). The Hawthorne effect may influence the results.

However, the subjects did to know which diet they were on:

Volunteers were not told the identity of the test cereal in the foods provided to them.

— (Bird et al. (2008), p. 1033)

Example 7.5 (Hawthorne effect) In an experimental study (Lorenz et al. 2019) to compare the efficacy of a new type of toothpaste, participants were given two types of toothpaste to use (a new type, and an exisiting type), and evaluations of plaque remaining on the teeth were taken. The authors state that:

… a plaque-reducing effect was seen not only in the test group but also in the control group. This phenomenon is due to the so-called Hawthorne effect that can lead to an overestimation of the effect and false positive results.

Lorenz et al. (2019), p. 5

That is, since all participants knew they were being assessed after brushing their teeth, there may have been a tendency to brush their teeth better than usual. The authors then state:

To minimize the Hawthorne effect, longer study durations of more than 6 months were suggested.

Lorenz et al. (2019), p. 6

7.4.3 Placebo effect

What if people thought they were on the wholegrain diet, but they weren’t? Perhaps surprisingly, individuals in a study may report effects of a treatment (either positive or negative), even if they have not received an active treatment. This is called the placebo effect.

Definition 7.4 (Placebo effect) The placebo effect is when individuals report perceived or actual effects without having received the treatment.

Managing the placebo effect is difficult! However, impact of the placebo effect can be minimized using:

  • Good planning: Experts may be able to suggest other ways exist to reduce this effect (e.g., measuring long-term effects rather than just short-term effects (Example 7.5) or using different designs (Kessels et al. 2019)).
  • Controls: Use units of analysis without the treatment applied, but as similar as possible in every other way to those units of analysis receiving the treatment. This allows the effect of the treatment to be ssessed, over and above the placebo effect.
Definition 7.5 (Control) A control is a unit of analysis without the treatment applied (but as similar as possible in every other way to other units of analysis).

Sometimes the control group receives a placebo. A placebo is a non-effective treatment. Those who receive the placebo should be selected through random allocation when possible. Sometimes, using a placebo is unethical. The Wikipedia entry about placebos is intriguing.

Definition 7.6 (Placebo) A placebo is a treatment with no intended effect or active ingredient.

Example 3.10 (Placebo effect) In the Himalaya 292 study, the authors report

On each day of the intervention periods, volunteers were asked to consume a combination of bread, breakfast cereal, muffins and crackers that would supply in total 103g of the test cereal. The aim was for each volunteer to consume 60g cereal flakes (or puffed rice for the refined cereal diet), two slices of bread, one muffin and six savoury crackers each day. Volunteers were not told the identity of the test cereal in the foods provided to them

— (Bird et al. (2008), p. 1033)

That is, the subjects were blinded to the diet they were exposed to. However, some may think they are on the refined cereal or Himalaya diet, and respond accordingly (perhaps unconsciously).
Think 7.2 (Controls) To test the effectiveness of a new drug, patients are to report to a GP to receive injections of a new drug. We wish to compare to people who do not get the injection. What is the control?
It is not just people who do not get the injections. Ideally, controls would be people who, like the treatment group, report to a GP and receive an injection… however, they just receive an injection that will do nothing.
Example 5.3 (Placebo effect) Three active analgesics (pain relievers) were compared to a placebo (Huskisson 1974). Four different coloured placebos were used. The most pain relief was experienced by those taking red placebos (Fig. 7.6), who experienced even more pain relief than those given true pain relievers.
Pain relief, for various pain relief medicine

FIGURE 7.6: Pain relief, for various pain relief medicine

Example 7.6 (Placebo effect) A study of placebos (Waber et al. 2008) gave half the subjects a placebo, but told them that the pill was an expensive (impling ‘very effective’) pain killer ($2.50 per tablet). The other half were also given a placebo, but were told that the pill was a discount (impling ‘less effective’) pain killer ($0.10 per tablet). About 85% of participants in the first group reported a pain reduction, yet only 61% in the second group reported a pain reduction. Remember that both groups actually received a placebo!

7.4.4 Observer effect

What if the researchers assessing the outcomes knew the diet allocated to each patient? Perhaps surprisingly, this can have an (unconscious) impact on the values of the response variable. This is called the observer effect (or, in experiments, sometimes called the experimenter effect).

Definition 7.7 (Observer effect) The observer effect is when the researchers unintentionally influence the behaviour of subjects.

The impact of the observer effect can be minimized using:

  • Good planning: Think about reasonable means for minimising the observer effect.
  • Blinding of the researchers: Blinding the researchers may be possible: that is, the people giving the treatment and the people evaluating the treatment do not know what treatment has been given. Instead, a third party can be used. For example, the researchers may give an assistant two drugs labelled A and B. The assistant then administers the drug and evaluates the participants’ response to the treatments. Later, the assistant tells the researchers whether Drug A or Drug B performed better, but only the researchers know what drugs the labels A and B refer to.

The observer effect does not just apply to situations where people are used as participants.

Example 7.7 (Observer effect) ‘Clever Hans’ (https://en.wikipedia.org/wiki/Clever_Hans) was a horse that seemed to be able to perform simple mental arithmetic. After much study, Carl Stumpf realised that the horse was responding to involuntary (and unconscious) cues from the trainer. This was discovered, in part, by using an experiment where the people interacting with the horse were blinded.

The same effect has been observed in narcotic sniffer dogs (Bambauer 2012), who may respond to their handlers’ unconscious cues.
The observer effect is about unconsciously influencing the outcome; that is, the researchers are not aware that it is occurring. When researchers intentionally influence the outcomes, this is called fraud (Sect. 4.3.2).

7.4.5 Describing blinding

Blinding is when those involved in the study do not know which comparison group the study individuals are in:

  • A study can blind the researcher to knowing what comparison group the study individuals are in.
  • A study can blind the participants to knowing what comparison group they are in.
  • A study can blind the analysts to knowing what comparison group the inidividuals are in during analysis.

If only the participants are blinded, the study is called single blind. If both the researchers and participants are blinded, the study is called double blind. If the researchers, participants and the analyst are blinded, the study is called triple blind. However, for clarity, explicitly stating who or what is blinded is best. Blinding should be considered in all studies, when possible (and it is not always possible).

Blinding of participants does not just apply to people; it is also relevant with animals (Example 7.7 about Clever Hans).

Think 7.3 (Blinding) Why might it be necessary to blind the analyst to the treatments being used?


Bambauer J. Defending the dog. Oregon Law Review. HeinOnline; 2012;91:1203.
Bird AR, Vuaran MS, King RA, Noakes M, Keogh J, Morell MK, et al. Wholegrain foods made from a novel high-amylose barley variety (Himalaya 292) improve indices of bowel health in human subjects. British Journal of Nutrition. 2008;99:1032–40.
Clark RL, Famodu OA, Holásková I, Infante AM, Murray PJ, Olfert IM, et al. Educational intervention improves fruit and vegetable intake in young adults with metabolic syndrome components. Nutrition Research. 2019;62:89–100.
Huskisson EC. Simple analgesics for arthritis. British Medical Journal. 1974;4:196–200.
Kessels R, Mozer R, Bloemers J. Methods for assessing and controlling placebo effects. Statistical Methods in Medical Research. SAGE Publications Sage UK: London, England; 2019;28(4):1141–56.
Lorenz K, Hoffmann T, Heumann C, Noack B. Effect of toothpaste containing amine fluoride and stannous chloride on the reduction of dental plaque and gingival inflammation. A randomized controlled 12-week home-use study. International Journal of Dental Hygiene. Wiley Online Library; 2019;
McCarney R, Warner J, Iliffe S, Van Haselen R, Griffin M, Fisher P. The hawthorne effect: A randomised, controlled trial. BMC medical research methodology. BioMed Central; 2007;7(1):30.
Miller EE, Boyle LN. Behavioral adaptations to lane keeping systems: Effects of exposure and withdrawal. Human factors. 2019;61(1):152–64.
Waber RL, Shiv B, Carmon Z, Ariely D. Commercial features of placebo and therapeutic. Journal of the American Medical Association. 2008;299(9):1016–7.