2 Research questions

In this chapter, you will learn to:

  • write operational and conceptual definitions.
  • ask quantitative research questions.
  • identify and write quantitative research questions.
  • identify the variables implied by a quantitative research question.
  • identify observational and experimental studies.
  • describe and identify the units of analysis and units of observations in a study.

2.1 Introduction

Asking clear and answerable research questions (RQs) is important, as the RQ impacts all other components of the research. Quantitative research summarises and analyses the data using numerical methods (like averages or percentages), so the RQ must be appropriate for analysis using quantitative methods.

Quantitative RQs must be written carefully so they can be answered effectively. In this book, four different types of RQs are studied:

  • descriptive RQs (Sect. 2.2);
  • relational RQs (Sect. 2.3);
  • repeated-measures RQs (Sect. 2.4);
  • correlational RQs (Sect. 2.9).

2.2 Descriptive RQs

All RQs identify a large group of interest to be studied, called a population (Sect. 2.2.1). Descriptive RQs study something about the population, called the outcome (Sect. 2.2.2).

2.2.1 The population

The population is any broad group of interest; for example:

  • all German males between \(18\) and \(35\) years of age.
  • all bamboo flooring materials manufactured in China.
  • all elderly females with glaucoma in Canada.
  • all Pinguicula grandiflora growing in Europe.

Definition 2.1 (Population) A population is a group of individuals from which the total set of observations of interest could be made, and to which the results will (hopefully) generalise.

Populations comprise many individuals, sometimes called cases. If the individuals are people, individuals are also called subjects.

The words population, individuals and cases do not just refer to people, though they may be commonly used that way in general conversation.

The data are rarely taken from all the individuals in the population: all individuals are rarely accessible in practice. For example, testing a new drug cannot possibly study all people (especially people not yet born who might use the drug). The population is 'all people', not just those people studied.

The population in a RQ is not just those studied; it is the whole group to which results could generalise.

In contrast, a sample is a subset of the population from which data are obtained (Chap. 5).

Definition 2.2 (Sample) A sample is a subset of individuals from the population from which data are collected.

Example 2.1 (Samples) A study of American college women (Woolf et al. 2009) aimed to assess iron status in highly-active women, and in sedentary women.

The sample comprises \(28\) active and \(28\) sedentary American college women, from which data are collected. The population is all active and sedentary American college women, not just the \(56\) in the study. The group of \(56\) subjects is the sample.

Completely and precisely defining the population sometimes requires refining or clarifying the population, using exclusion and/or inclusion criteria. Exclusion and inclusion criteria clarify which individuals are explicitly included or excluded from the population. Exclusion and inclusion criteria should be explained when their purpose is not obvious. Exclusion and inclusion criteria are not necessary; none, one or both may be used.

Definition 2.3 (Inclusion and exclusion criteria) Inclusion criteria are characteristics that individuals must meet explicitly to be included in the study.

Exclusion criteria are characteristics that explicitly disqualify potential individuals from being included in the study.

Example 2.2 (Inclusion and exclusion criteria) Concrete test cylinders with cracks may be excluded from strength tests.

People with severe asthma may be excluded from exercise studies.

A study on the influenza vaccine (Kheok et al. 2008) listed the Population as 'health-care workers' (Kheok et al. 2008, 466), and the sample comprised healthcare workers at two specific hospitals. The population was refined using exclusion criteria: those (p. 466)

...declining to give consent, a history of egg protein allergy, and neurological or immunological conditions that are contraindications to the influenza vaccine.

Example 2.3 (Inclusion and exclusion criteria) A study (Guirao et al. 2017) of the walking abilities of amputees used inclusion and exclusion criteria. Inclusion criteria included (p. 27):

... length of the femur of the amputated limb of at least \(15\) cm measured from the greater trochanter; use of the prosthesis for at least \(12\) months prior to enrollment and more than \(6\) h/day...

Exclusion criteria included (p. 27):

... the presence of cognitive impairment hindering the ability to follow instructions and/or perform the tests; body weight over \(100\) kg...

2.2.2 The outcome

Descriptive RQs study something about the population, called the outcome. Because the RQ concerns a large group (the population), the outcome must be suitable for describing a group (not individuals). Hence, the outcome is a numerical quantity (such as an average or percentage) that can summarise a group of individuals.

Definition 2.4 (Outcome) The outcome in a RQ is the result, output, consequence or effect of interest in a study, numerically summarised for the population.

The outcome of interest in a population may be (for example) the

  • average increase in heart rates after \(30\) mins of exercise.
  • average amount of wear after \(1000\) hrs of use.
  • proportion of people whose pupils dilate.
  • average weight loss after three weeks on a diet.
  • percentage of seedlings that die.

The outcome in a RQ summarises a population; it does not describe the individuals in the population.

2.2.3 Purposes of RQs

Depending on the purpose of the study, descriptive RQs have one of these forms:

  • Estimation: Among {the population}, what is {the outcome}?
  • Decision-making: Among {the population}, is {the outcome} equal to {a given value}?

These are not 'recipes', but guidelines.

Answering estimation descriptive RQs is studied in Chaps. 24 and 25. Answering decision-making descriptive RQs is studied in Chaps. 31 and 32.

Example 2.4 (Descriptive RQs) A study examined the 'body temperature of \(148\) healthy men and women' (Mackowiak, Wasserman, and Levine 1992) aged between \(18\) to \(40\); this is the population. The outcome of interest in this population is the average body temperature.

One descriptive RQ was:

What is the average body temperature?

This is an estimation RQ. A decision-making descriptive RQ they also studied was whether the average body temperature was the value that had been commonly accepted by medical professionals:

Is the average body temperature really \(98.6^\circ\)F (\(37.0^\circ\)C)?

2.3 Relational RQs

Usually studying relationships is more interesting than simply describing a population. Relational RQs compare the outcome for different groups of individuals in the population. These comparisons are called between-individuals comparisons, as they compare the outcome between different groups of individuals.

Examples include:

  • Comparing the average amount of wear in floor boards between two groups of boards: standard wooden flooring materials, and bamboo flooring.
  • Comparing the average heart rates across three groups of people: those who received no dose of a drug, those who received a daily dose of the drug, and those who received a twice-daily dose of the drug.

Definition 2.5 (Comparisons (between individuals)) The between-individuals comparison in the RQ identifies the small number of different groups of individuals for which the outcome is compared.

Example 2.5 (Between-individuals comparison) A study by Bird et al. (2008) gave one group of participants a diet using refined flour, and gave another group of participants a diet using a new flour variety. Faecal weight was compared for the two groups. The type of diet is the between-individuals comparison.

Relational RQs have a between-individual comparisons.

Typically, relational RQs have one of these forms:

  • Estimation: Among {the population}, what is the difference in {the outcome} for {the alternatives being compared}?
  • Decision-making: Among {the population}, is {the outcome} the same for {the alternatives being compared}?

Example 2.6 (Relational RQs) Consider this RQ (based on Estévez-Báez et al. (2019)):

Among Cubans between \(13\) and \(20\) years of age, is the average heart rate the same for females and males?

The population is 'Cubans \(13\) and \(20\) years of age', the outcome is 'average heart rate', and the between-individuals comparison is between two separate groups: 'between females and males'. This is a relational RQ.

This RQ is a decision-making RQ, since it asks if the average heart rate is the same for females and males. An estimation-type relational RQ would ask about the size of difference in the average heart rate between females and males.

2.4 Repeated measures RQs

Rather than comparing the outcome for different groups of individuals, repeated measures RQs compare the outcome multiple times within the same individuals.

These comparisons are called within-individuals comparisons, as they compare the outcome within the same individuals, not across different groups of individuals. The multiple measurements may be different points in time (e.g., the height of the same trees at one, two and five years after planting), but do not have to be time points.

Examples include:

  • Comparing the strength of hind legs of horses to the fore legs of the same horses.
  • Comparing the thickness of the cornea in eyes with glaucoma and eyes without glaucoma, in people with one glaucoma eye.
  • Comparing the average amount of wear in many individual floor boards after one, five and ten years of use.

Definition 2.6 (Within-individuals comparison) The within-individuals comparison in the RQ identifies the small number of different, distinct situations for which the outcome is compared multiple times for each individual.

Example 2.7 (Between- and within-individual comparisons) Consider comparing left- and right-legs strengths of professional football players.

A between-individuals comparison would compare the left and right leg strengths between different groups of footballers: one group would have their left-leg strength measured, and the other would have their right-leg strength measured. This study has a between-individuals comparison.

In contrast, a similar study measures both left- and right-leg strength on the same individuals. This study examines within-individuals changes: the differences between the left- and right-leg strength within each individual. In this study, no between-individuals comparison exists, as different groups are not being compared.

Studies may use both within- and between-individuals comparisons (see Sect. 35.9). For instance, a study may examine the change in each individuals blood pressure (the within-individuals comparison), for two drugs given to different groups (the between-groups comparison).

Repeated measures RQs have a within-individual comparisons.

Typically, repeated measures RQs have one of these forms:

  • Estimation: Among {the population}, what is the change in {the outcome} for {the alternatives being compared}?
  • Decision-making: Among {the population}, is {there a change in the outcome} for {the alternatives being compared}?

Example 2.8 (Within-individuals relational RQ) To understand tree-dwelling marsupials, a study compared the temperature in the same tree hollows in both summer and winter (Rowland, Briscoe, and Handasyde 2017):

For tree hollows in the Strathbogie Ranges, Australia, what is the mean temperature difference between summer and winter?

The comparison is within individuals, as the temperature is measured for the same tree hollows at the two times. This is a relational, estimation-type relational RQ.

Repeated-measures RQs with only two comparisons are often called paired.

Example 2.9 (Paired repeated-measures study) D. A. Levitsky, Halbmaier, and Mrdjenovic (2004) conducted a study where the weights of university students were recorded both at the beginning university, and then after \(12\) weeks. The comparison is within individuals, and the study is a repeated measures study. Since each student has a pair of weight measurements, this is a paired study.

2.5 Interventions

In some studies, the comparison naturally occurs (e.g., age group of people), and in some studies the comparison is manipulated by researchers and called an intervention (e.g., the amount of fertilizer applied).

Definition 2.7 (Intervention) An intervention is present when researchers manipulate (or impose) the comparison to determine the impact on the outcome.

Comparisons that are not manipulated by the researchers are called conditions. Comparison that are manipulated by the researchers, or imposed on the individuals by the researchers, are called treatments

Between- or within-individuals comparisons can be an intervention. Within any type of comparison, the analysis is the same whether an intervention is used or not. However, the interpretation of the results depend on whether an intervention is used or not (Sect. 3.6).

An intervention is present when the researchers:

  • explicitly give doses of a new drug to patients.
  • explicitly apply wear testing loads to two different flooring materials.
  • explicitly expose people to different stimuli.
  • explicitly apply different doses of fertiliser.

Example 2.10 (Intervention) A study by Bird et al. (2008) supplied one group of participants a diet using refined flour, and supplied another group of participants a diet using a new flour variety. The type of diet is the between-individuals comparison. Since the researchers manipulate which subjects ate which flour, this study has an intervention.

Example 2.11 (No intervention) A study comparing the average blood pressure in female and male Scots measured blood pressure using a blood pressure machine (a sphygmomanometer). The research team interacts with the participants and uses the machine to measure blood pressure, but there is no intervention. Using the sphygmomanometer is just a way to measure blood pressure, to obtain the data.

There is no intervention: the comparison is between females and males, which cannot be manipulated or imposed on the individuals by the researchers.

A study of American college women (Woolf et al. 2009) measured iron status in highly-active and sedentary women. What is the outcome, comparison (if any), and intervention (if any)?

Outcome: 'average iron status' (which would need an operational definition.)

Comparison: between two groups of individuals: highly active and sedentary women (i.e., between individuals). These terms would also need operational definitions!

Intervention: Probably none; an intervention would mean the researchers tell each individual woman to be highly active or sedentary, which seems unlikely.

Often, one of the comparison groups is the control group. The control group is a comparison group not receiving the treatment being studied, or not having the condition being studied, but as similar as possible to the other units of analysis in all other ways. The control group acts like a benchmark for detecting changes in the outcome (Sect. 7.6). Sometimes the control group receives a placebo: a non-effective treatment that appears to be the real treatment.

Definition 2.8 (Control) A control is an individual without the treatment or condition of interest, but as similar as possible in every other way to other individuals.

Definition 2.9 (Placebo) A placebo is a treatment with no intended effect or active ingredient, but appears to be the real treatment.

Example 2.12 (Control group) To test the effectiveness of a new medication, patients report to a doctor to receive injections of the new drug. Some patients are assigned to the control group. The controls are not simply people who don't get the injections. Ideally, controls would be people who, like the treatment group, report to a doctor and receive an injection... however, the injection is ineffective (a placebo).

Together, the Population, Outcome, Comparison and Intervention form the POCI acronym (sometimes written as the PICO acronym) to aid remembering the elements of RQs.

2.6 Two purposes of RQs

As noted earlier, RQs can also be written with one of two purposes in mind. Estimation RQs ask how precisely a value in the population is estimated by using the sample, and are answered using confidence intervals. Answering estimation RQs are discussed in this book in Chaps. 24 to 29, and Sect. 39.6.

Decisions-making RQs are concerned with making a decision about a population, and are answered using hypothesis testing. Answering decision-making RQs are discussed in this book in Chaps. 31 to 36, plus Sects. 39.7 and 38.2.

Example 2.13 (Decision-making RQs) A study (Thane, Bates, and Prentice 2004) of 'British young people aged \(4\)--\(18\)' asked and answered numerous RQs. One relational RQ is:

In British young people aged \(4\)--\(18\), is the average zinc intake the same for boys and girls?

This is an decision-making RQ.

Decision-making RQ have two possible answers. For the example above: the average zinc intake either is the same for boys and girls, or is not the same for boys and girls (Fig. 2.1). However, answers are rarely clear in practice, since only one of the countless possible samples from the population are studied. Instead, researchers decide how much sample evidence support a particular hypothesis about the population.

Evidence may support or contradict a hypothesis; evidence rarely proves a hypothesis (at least, without any other support, such as theoretical support). Ultimately, after collecting data from a sample, a decision must be made about which explanation about the population is more consistent with the data collected.

Two possible answers to the RQ about zinc intake

FIGURE 2.1: Two possible answers to the RQ about zinc intake

Decision-making RQs can be asked in different ways. For the zinc-intake study above (Fig. 2.1), the RQ could ask if:

  • the average zinc intake is the same for boys and girls?
  • the average zinc intake is different for boys and girls?
  • the average zinc intake is lower for boys compared to girls?
  • the average zinc intake is higher for boys compared to girls?

The first two are two-tailed RQs (and are essentially asking the same thing), since they ask about a difference either way: the average zinc intake could be higher for girls or higher for boys; we are just interested in whether any difference is present. The last two are one-tailed RQ, since they ask specifically about a difference in just one direction: girls greater than boys, or boys greater than girls.

Most RQs are two tailed, unless a good reason exists to ask a one-tailed RQ (e.g., a drug has been developed specifically to reduce blood pressure). One of the last two would only be adopted if there was a specific reason why a one-tailed direction was suspected before the data were collected.

In general, RQs should be written as two-tailed RQs, unless a good (and justifiable) reason exists for asking a one-tailed question.

2.7 Writing RQs: an example

Suppose you notice some people taking echinacea (a herb) when they get a common cold. You may wonder: does taking echinacea help with a cold? This may lead to an initial RQ:

Is it better to take echinacea when you have a cold?

This RQ is clearly poor, but is a starting point. This RQ can be refined by clarifying the POCI elements. For example, what population could we study? Many options exist: All Australians, or Australian adults in a specific location. Some of these may not be practical (i.e., when a sample cannot easily be obtained from the population).

What outcome could be used to to determine echinacea's effectiveness? Many options exist, such as the average cold duration, or the percentage of people who take days off work.

The initial RQ is also vague: better than what? The outcome could be compared between groups (such as those taking echinacea and those who do not). A within-individuals comparison seems unsuitable for this RQ.

The study could also have intervention or not, which has implications for how the study is conducted and how the results are interpreted. If the study did not have an intervention, the subjects would decide for themselves how to treat their cold. If the study did have an intervention, various interventions could be imposed by the researchers; the dose frequency or the doses amounts could be imposed, for instance.

The P, O, C and I do not have to be comprehensively described in the RQ; some information could be provided later as operational definitions (e.g., dose) and using exclusion and inclusion criteria (e.g., exclude teens with chronic health conditions).

Many terms need defining, too: What is meant by 'echinacea' (fresh? tablet form? as a tea?); 'cold' (self-diagnosed? diagnosed by a doctor?), and so on.

The following short video may help explain some of these concepts:

2.8 Variables: from populations to individuals

Based on the above, consider this RQ (based on Barrett et al. (2010)):

Among Australian teenagers with a common cold, is the average duration of cold symptoms shorter for teens given a daily dose of echinacea compared to teenagers taking no echincea?

This is an one-tailed relational RQ. However, the data to answer this RQ come from a sample of individuals in that population. Each piece of information obtained from or about each individual is called a variable, because the values can vary from individual to individual.

Definition 2.10 (Variable) A variable is a single aspect or characteristic associated with the individuals, whose values can vary from individual to individual.

Examples of variables include: the duration of cold symptoms, gender, place of birth, or hair colour. The RQ identifies the variables needed to answer the RQ, though other variables may be (and typically are) measured also (Sect. 6.4).

A variable is a single aspect that can vary from individual to individual. While your city of birth may not change, 'city of birth' is a variable because it varies from individual to individual.

Example 2.14 (Variables) 'Duration of cold symptoms' is a variable: it is obtained from individuals, and its value can vary from individual to individual.

The 'average duration of cold symptoms' is the outcome, a numerical summary of many individuals' cold durations.

While many variables can be recorded from individuals in the population, two essential variables are (Table 2.1):

  • The response variable, which records information to determine the outcome.
  • The explanatory variable, which records information to determine the comparison.

The value of the response variable may change in response to the value of the explanatory variable. The value of the explanatory variable may explain the value of the response variable.

Definition 2.11 (Explanatory variable) An explanatory variable may (partially) explain or cause changes in another variable of interest (the response variable).

Definition 2.12 (Response variable) A response variable records the result, output, consequence or effect of interest from changes in another variable (the explanatory variable).

The response variable is sometimes called the dependent variable, and the explanatory variable is sometimes called the independent variable.

We avoid these terms, since the words 'dependent' and 'independent' have many different meanings in research.

TABLE 2.1: The relationship between the population and the individuals
Population Individuals
Outcome: \(\rightarrow\) Response variable
Comparison: \(\rightarrow\) Explanatory variable

The RQ cannot be answered without information about these two variables. The outcome is a summary of the values of the response variable (Table  2.2) recorded from many individuals. Similarly, the values of the explanatory variable measured on the individuals distinguish between the values of the comparison (Table 2.3 and  2.4) being made.

The POCI elements, and two important variables

FIGURE 2.2: The POCI elements, and two important variables

TABLE 2.2: Examples of the outcome and the corresponding response variable
\(\rightarrow\)
Outcome describing the population \(\rightarrow\) Response variable in individuals
Average increase in diastolic blood pressure, from before to after exercise \(\rightarrow\) Increase in diastolic blood pressure of individuals, from before to after exercise
Percentage of seedlings that sprout \(\rightarrow\) Whether or not an individual seedling sprouts
Proportion owning iPad \(\rightarrow\) Whether or not an individual owns an iPad
Average cold duration \(\rightarrow\) Cold duration for individuals
Percentage of concrete cylinders having fissures \(\rightarrow\) Whether or not an individual cylinder has fissures
TABLE 2.3: Examples of the between-individuals comparison and the corresponding explanatory variable
Comparison being made Explanatory variable in individuals
Between jarrah, beech and bamboo floor boards \(\rightarrow\) Type of floorboard in homes
Between \(30\) kg/ha and \(40\) kg/ha fertilizer rates \(\rightarrow\) Application rate in paddocks
Between people in \(20\)s, \(30\)s and \(40\)s \(\rightarrow\) Age group for each person
TABLE 2.4: Examples of the within-individuals comparison and the corresponding explanatory variable
Comparison being made Explanatory variable in individuals
Before and after \(\rightarrow\) Time
Between left and right arms \(\rightarrow\) Which arm
Between fore legs and hind legs \(\rightarrow\) Which legs

Example 2.15 (Variables) For the final RQ for the echinacea study (Sect. 2.7), 'the duration of cold symptoms' is the response variable, and 'whether echinacea is taken or not' is the explanatory variable. The type of medication is taken before the cold symptoms disappear, and may even explain the duration of the cold symptoms.


Free Online Poll Maker

The Population is 'carrots grown in Buderim' 8 weeks after planting. From these carrots, we need to collect Whether or not Thrive was applied and the weight of the carrots \(8\) weeks after planting.

The response variable is 'the weight of each individual carrot \(8\) weeks after planting', and the explanatory variable is 'whether or not Thrive was used on each carrot'.

('The number of carrots planted' is not even a variable: it is not information recorded about the individuals, but a summary of information.)

Consider this RQ:

For overweight men over \(60\), is the average weight loss after three weeks the same for a diet high in fresh fruit and a diet high in dried fruit?

The outcome is the average weight loss; the response variable is the weight loss for each individual man. (This would be found by measuring their weight before and after three weeks on the diets; it is measured within-individuals.)

The between-individuals comparison is between the two diets; the explanatory variable is the diet each man is on.

Example 2.16 (Variables) A study compared the average lead concentration in the surfaces of public playgrounds in Boston.

The population comprises public playgrounds in Boston; each public playground is an individual. The outcome is the average lead concentration in the surfaces over many playgrounds; the response variable is the lead concentration in the individual playground surfaces.

The between-individuals comparison is between the four types of surfaces (rubber, soil, sand and mulch). The explanatory variable is the type of surface.

2.9 Correlational RQs

A different type of RQ is a correlational RQ. Correlational RQs are not concerned with summarising an outcome over a population. Instead, correlational RQs explore relationships between two variables measured or observed on the individuals. Typically, correlational RQs have the form:

  • Estimation: Among {the population}, how strong is the relationship between {the outcome} and {something else}?
  • Decision making: Among {the population}, is {the outcome} related to {something else}?

Examples include:

  • Studying the relationship between heart rate and the number of grams of caffeine consumed that day.
  • Studying the relationship between the height of plants and the number of hours of sunlight per day.

Example 2.17 (Correlational RQs) The study in Example 2.6), about Cubans and heart rates, could also have asked:

Among Cubans between \(13\) and \(20\) years of age, is the heart rate related to age?

This RQ is a decision-making correlational RQ, since it asks if the heart rate is related to age. An estimation corelational RQ might be:

Among Cubans between \(13\) and \(20\) years of age, how strong is the relationship between average heart rate and age?

Example 2.18 (Correlational RQs) The Wollemi pine was discovered by science in 1994. Offord and Zimmer (2023) studied the growth of these rare plants.

One correlational RQ they studied was the relationship between the size of trees (using DBH, diameter at breast height), and the pH of the soil. The two variables are the DBH and pH, both recorded for many trees.

In addition, they studied the relationship between the DBH for each tree at at various times after the planting date was recorded. Each tree has the DBH measured over time, for many time points. Time is the within-individuals comparison.

In some (but not all) situations, one of the variables can be considered as possibly influencing the value of the other variable. In these situations, one variables can be called the explanatory variable (which may explain changes in the other variable) and the other the response variable (whose values respond to changes in the explanatory variable).

Example 2.19 (Correlational RQs) Consider studying marathon runners. A RQ exploring the relationship between the daily water intake and the daily amount of sun exposure for many individuals would be a correlational RQ.

The daily amount of sun exposure is the explanatory variable, and the daily water intake is the response variable. It seems possible that the amount of sun exposure would influence the water intake.

2.10 Units of observation and analysis

Units of observation and units of analysis are different, but similar, concepts that must be distinguished to properly identify a population.

Consider this descriptive RQ:

In English \(20\)-something men, what is the average thickness of head-hair strands?

Answering the question by measuring \(100\) hair strands from one man is problematic: only one man is represented. While there are \(100\) observations, all the data come from one man; little is learnt about \(20\)-something men in general. Instead, a lot is learnt about one specific man. The population is represented by just one man.

In this study, each individual hair is a unit of observation: the hair strands are measured to obtain 'thickness of head-hair strands'.

Definition 2.13 (Unit of observation) Unit of observation: The 'who' or 'what' which are observed, from which measurements are taken and data collected.

Since each hair comes from the same man, each of those hairs have essentially 'lived their life together': they are washed at the same time, with the same shampoo, exposed to the same amount of sunlight and exercise, share genetics, etc. However, different men would potentially use different shampoo, exercise differently, have different genetics, and so on. Each man tends to be different, and lives differently and independently of others.

Each man is a collection of units of observations (hair strands). This leads to a similar, but different, concept: the unit of analysis. In the example above, the man is a unit of analysis, and provides \(100\) observations.

Definition 2.14 (Unit of analysis) Unit of analysis: The smallest collection of units of observations (and perhaps the units of observations themselves) about which generalizations and conclusions are made; the smallest independent 'who' or 'what' for which information is analysed.

Importantly, the sample size for the study is the number units of analysis. In the hair-thickness study, a person is a unit of analysis, so the sample size is just one. Only one example of the population of men is in the study. Each unit of analysis (man) has \(100\) units of observation (hair strands).

The number of units of analysis in a study is the size of the sample.

Example 2.20 (Units of analysis) In the hair-strand study, each hair strand is a unit of observation: hair strand thicknesses are taken from individual hair strands. The unit of analysis is the person: the hair strands from each man share much in common. 'Men' operate independently, but the hairs on each man are not independent entities.

The individuals are the units of analysis.

Example 2.21 (Units of analysis, observation) An article on the Spectrum website (accessed 18 Nov 2022) reported a study where researchers 'looked at \(10\) neurons from each of the \(16\) mice' in the study. The researchers treated each neuron as an independent observation, giving a sample size of \(n = 16\times 10 = 160\).

However, neurons in the brain of the same animal are not independent observations. The unit of analysis is the mouse, and the unit of observation is the neuron. The actual sample size was \(n = 16\); each unit of analysis has \(10\) units of observation.

A total of \(160\) neurons from 16 mice is very different to a study of \(160\) neurons from \(160\) genetically-different mice.

The units of observation and units of analysis may be the same, and often are the same. However, they are sometimes different, and identifying these situations is crucial. Importantly, studies compare units of analysis, not units of observation.

Sometimes the units of analysis and units of observation are the same.

Example 2.22 (Units of analysis) A study compared two physical activity (PA) programs. Each of \(44\) children in the study was allocated to one of two PA programs (with parental agreement). The children's fitness was measured for every student at the end of the six-month study.

The units of observation are the individual students, as the fitness measurements are taken from each students. The units of analysis are also the individual students, as the PA program was allocated to each student individually, and each student has their own sport, family routines and activities, etc. Each unit of analysis (student) has one unit of observation.

There are \(44\) units of analysis, each with one unit of observation.

Example 2.23 (Units of analysis) Consider comparing the percentage of females and males wearing sunglasses at a specific beach.

People in a group at the beach will probably not be operating independently: people in groups tend to behave similarly. For example, a couple will often both be wearing or both not wearing sunglasses.

The researchers may decide not to use data from groups, and only gather data from individuals. Alternatively, the researchers may decide to use people groups as the unit of analysis (some will be groups of one), and record data from just one person in any group (ideally specifying before-hand from which group member to take data; e.g., the person closest to the researchers when the group is noticed).


Free Online Poll Maker


Free Online Poll Maker

Units of observation: the individual students, as the fitness measurements are taken from the students individually.

Units of analysis: the schools, as the PA program was allocated to each school. All students at School A are exposed Program 1, but all students at School A are also likely to be exposed to similar weather, fitness opportunities, physical conditions, teachers and school-based philosophies, and so on.

The improvement in the children's fitness levels and the program are both variables.

The following short video may help explain some of these concepts:

2.11 Definitions

Research studies usually include terms that must be carefully and precisely defined, so that others know exactly what words and terms mean, without ambiguity. Two types of definitions can be given when necessary.

Definition 2.15 (Conceptual definition) A conceptual definition articulates precisely what words or phrases mean in a study.

Definition 2.16 (Operational definition) An operational definition articulates exactly how something will be identified, measured, observed or assessed.

In many cases, a clear operational definition is needed to describe how data will be collected to ensure repeatability and consistent data collection, by removing any ambiguity about how data are obtained.

Example 2.24 (Operational and conceptual definitions) Consider a study examining stress in students. A conceptual definition would describe what is meant by 'stress' (in contrast to, say, 'anxiety'). An operational definition would describe how 'stress' is measured, since stress cannot be measured directly (like height, for example).

'Stress' could be measured using a questionnaire or measuring physical characteristics, for instance. Other ways of measuring stress are also possible, and all have advantages and disadvantages.

Sometimes the definitions themselves are not important, provided a clear definition is given. However, to avoid confusion, commonly-accepted definitions should be used unless good reasons exist for using a different definition. When a commonly-accepted definition does not exist, the definition being used should be very clearly articulated, and the reason given if necessary.

Example 2.25 (Operational and conceptual definitions) A research article (Gillet et al. 2018) entitled 'Shoulder range of motion and strength in young competitive tennis players with and without history of shoulder problems' provided these necessary conceptual definitions (among others):

  • Young: \(8\)--\(15\) years;
  • Competitive tennis players: Some of the best players in their age category in France, and members of a French tennis centre of excellence.

An operational definition was provided for 'Shoulder strength': as measured using a hand-held dynamometer.

Players, administrators and fans are wary of concussions and head injuries in sport. A conference on concussion in sport developed this conceptual definition (McCrory et al. 2013):

... a complex pathophysiological process affecting the brain, induced by biomechanical forces...

However, an operational definition is needed to explain how to identify a player with concussion during a game. Rugby decided on this operational definition (Raftery et al. 2016):

... a concussion applies with any of the following:

  1. The presence, pitch side, of any Criteria Set 1 signs or symptoms (table 1)... [this table includes symptoms such as 'convulsion', 'clearly dazed', etc.];

  2. An abnormal post game, same day assessment...;

  3. An abnormal \(36\)--\(48\) h assessment...;

  4. The presence of clinical suspicion by the treating doctor at any time...

Example 2.26 (Operational and conceptual definitions) Consider a study requiring water temperature to be measured.

An operational definition would explain how the temperature is measured: the thermometer type, how the thermometer was positioned, how long was it left in the water, and so on.

In contrast, a conceptual definition might describe the scientific definition of temperature (and would not be needed, as 'temperature' is a well-understood term).

A study of snacking in Australia (Fayet-Moore et al. 2017) used this conceptual definition of an 'eating occasion':

...one or more food or beverage items consumed at the same time of day...

and a 'snacking occasion' as

...one or more food or beverage items consumed at the same time of day within a snacking time period...

Finally then, 'snacking' was defined as:

Eating occasions that occurred during breakfast, midday and evening meals were meals and all eating occasions that occurred between these meals were classified as snacking.

These are all conceptual definitions, explaining what the terms mean.

An operational definition would explain how the data were obtained from the participants (e.g., using a food diary).

Meline (2006) discusses five studies about stuttering, each using a different operational definition:

  • Study 1: As diagnosed by speech-language pathologist.
  • Study 2: Within-word disfluences greater than \(5\) per \(150\) words.
  • Study 3: Unnatural hesitation, interjections, restarted or incomplete phrases, etc.
  • Study 4: More than \(3\) stuttered words per minute.
  • Study 5: State guidelines for fluency disorders.

People may be classified as stutterers by some definitions but not others, so it is important to know which definition is used.

A study examined the possible relationship between the 'pace of life' and the incidence of heart disease (Levine 1990) in \(36\) US cities.

The researchers used four different operational definitions for 'pace of life' (remember the article was published in 1990!):

  1. The walking speed of randomly chosen pedestrians.
  2. The speed with which bank clerks gave 'change for two $20 bills or [gave] two $20 bills for change'.
  3. The talking speed of postal clerks.
  4. The proportion of men and women wearing a wristwatch.

None of these perfectly measure 'pace of life', of course. Nonetheless, the researchers found that, compared to people on the West Coast,

... people in the Northeast walk faster, make change faster, talk faster and are more likely to wear a watch...

--- Levine (1990) (p. 455)

Define a 'smoker'.

This is very difficult!

Some studies use the categories Never smoked, Past smoker, and Current smoker... or ask people to self-identify as a smoker or not.

2.12 Preparing software

Most statistical software packages (including jamovi) use the same approach for organising the data (though exceptions exist for some types of analyses):

  • Each row represents one unit of analysis: the number of rows equals the number of units of analysis.
  • Each column represents one variable: the number of columns equals the number of variables. (An additional column of identifying information may also appear, such as the person's name, or concrete batch number.)

In statistical software, the variable names are not placed in a row (say, in Row 1, above the data itself), which might happen when using a spreadsheet. The names of the variables are the names of the columns.

Example 2.27 (Preparing statistical software) In Sect. 2.8, a RQ was asked about whether using echinacea or not reduced the duration of the common cold.

For this RQ, the variables are 'Duration of cold symptoms' (response), and 'Type of treatment' (explanatory); see Examples 2.14 and 2.15.

The number of rows in the data worksheet will equal the number of people in the study, since the person is the unit of analysis. The data worksheet needs at least two columns (Fig. 2.3, left panel):

  • one for duration of each individual's cold symptoms (say, Duration);
  • one for whether the individual received a dose of echinacea or received no medication (say, Treatment).

There may be an additional column recording the name or ID of each individual, and more columns recording other variables (such as age and height).

Example 2.28 (Preparing statistical software) Example 2.9 discussed a study (D. A. Levitsky, Halbmaier, and Mrdjenovic 2004) where the weights of university students were recorded both at the beginning university, and then after \(12\) weeks. The number of rows in the data worksheet will equal the number of people in the study, since the person is the unit of analysis. The data worksheet needs at least two columns (Fig. 2.3, right panel):

  • one for the student's weight at the start of university (say, Week 1);
  • one for the student's weight after \(12\) weeks at university (say, Week 12).
jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.

FIGURE 2.3: jamovi prepared for the data, with some data entered, and the variable names as the column headers. Left: a between-individuals comparison. Right: a within-individuals comparison.

2.13 Chapter summary

In this chapter, you have learnt to write research questions for quantitative analysis. All research questions (RQs) study some population (P). Descriptive RQs study some outcome (O) in the population. Relational RQs compare the outcome between different groups of individuals (a between-individuals comparison). Repeated-measures RQs compare the same outcome when measured on the same individuals multiple times (a within-individuals comparison). Some RQs also have an intervention (I): when the values of the comparison can be manipulated by the researchers. Correlational RQs ask about the relationship between variables in the population.

RQs may take one of two forms: Decision-making RQs (which may be one- ot two-tailed) or Estimation RQs.

For quantitative RQs, the outcome numerically summarises the population (or subsets of the population), so is usually worded in terms of percentages, averages, etc.

Data comes from individuals in the population by measuring, observing or assessing the response (or dependent) variable. The outcome is a numerical summary of the values of the response variable from many individuals. Similarly, the data concerning the comparison comes from measuring or observing the values of the explanatory (or independent) variables from individuals.

The who or what that observations are made from are called the units of observation. The smallest independent collections of units of observations (that is, units with very little in common) are called the units of analysis.

The following short video may help explain some of these concepts:

2.14 Quick review questions

Consider this RQ:

In elite female netball players, do players in defence positions have the same average number of knee injuries (per player, per season) than players in attacking positions?

  1. What is the comparison in this RQ?
  2. What type of comparison appears in this RQ: between or within individuals?
  3. What is the outcome?
  4. What is the response variable?
  5. What is the unit of analysis?
  6. What is the unit of observation?
  7. Is this RQ descriptive, relational, repeated measures or correlational?
  8. Is this RQ a decision-making or estimation question?
  9. If decision-making, is this RQ one- or two-tailed?

2.15 Exercises

Selected answers are available in App. E.

Exercise 2.1 For the following response variables, what are the corresponding outcomes?

  1. Whether a vehicle crashes or not.
  2. The height people can jump.
  3. The number of tomatoes per plant.

Exercise 2.2 For the following response variables, what are the corresponding outcomes?

  1. Whether or not a person owns a car.
  2. The time it takes for seedlings to sprout.
  3. The amount of caffeine in soft drinks.

Exercise 2.3 For the following comparisons, what are the corresponding explanatory variables?

  1. Between vegans and vegetarians.
  2. Between caffeinated and decaffeinated coffee.
  3. Between taking zero, one or two \(7\) mg iron tablet per day.

Exercise 2.4 For the following comparisons, what are the corresponding explanatory variables?

  1. Between frozen vegetables and fresh vegetables.
  2. Between \(91\) octane, \(95\) octane, and ethanol-blended car fuel.
  3. Between large cities and small cities.

Exercise 2.5 For the following studies, determine which have a between-individuals comparison and which have a within-individuals comparison. In each case, identify the outcome.

  1. A study to determine if a higher percentage of people at a particular city park wear hats in winter compared to summer.
  2. A study to determine if the average yield of a specific variety of tomato plants is the same when three different fertilisers are applied.
  3. A study to determine if a person's average balance-time on their right leg is the same as on their left leg.

Exercise 2.6 For the following studies, determine which have a between-individuals comparison and which have a within-individuals comparison. In each case, identify the outcome.

  1. A study to determine if average cholesterol levels are the same when measured on the same people before and after a diet change.
  2. A study to determine the change in the height of pine trees from year one to year three.
  3. A study to determine the temperature of tree hollows used for nesting in both summer and winter.

Exercise 2.7 A study of Phu Quoc Ridgeback dogs (Canis familiaris) explored the relationship between body length and body height (Quan, Tran, and Chung 2017).

  1. What type of RQ would be asked about the dogs?
  2. What are the response and explanatory variables?

Exercise 2.8 A study of typing speeds (Pinet et al. 2022) recorded typing speed and age for \(1301\) students. Is there evidence of a relationship between the mean typing speed and mean accuracy?

  1. What type of RQ would be asked about the dogs?
  2. What are the response and explanatory variables?

Exercise 2.9 Consider this RQ:

Among Danish university students, is the average resting diastolic blood pressure the same for students who regularly drive to university and those who regularly ride their bicycles to university?

  1. For this RQ, identify the population, outcome, and comparison (if any).
  2. For this RQ, is there an intervention? Explain.
  3. What type of RQ is this: estimation or decision-making?
  4. What operational and conceptual definitions would be needed?
  5. What information must be collected from each individual to answer the RQ (i.e., the variables)?
  6. Identify the units of analysis and the units of observation.

Exercise 2.10 Consider this article extract (Checkley et al. (2002), p. 210):

We conducted a \(4\)-year (1995--1998) field study in a Peruvian peri-urban community... to examine the relation between diarrhoea and nutritional status in \(230\) children \(< 3\) years of age

For this study:

  1. Identify POCI.
  2. Infer the primary research question.
  3. What type of question is used (descriptive; relational; repeated measures; correlational)?
  4. What operational definitions would be needed?
  5. What are the response and explanatory variables?

Exercise 2.11 Consider this RQ: 'Is the average walking speed the same when texting and talking on a mobile phone?'

  1. What type of question is used (descriptive; relational; repeated measures; correlational)? Is this RQ one- or two-tailed?
  2. Is there an intervention?
  3. What is the explanatory variable?
  4. What is the response variable?
  5. What is the outcome?

Exercise 2.12 Animals in an experiment are divided into pens (three animals per pen), and feed is allocated to each pen (Sterndale et al. 2017). Animals in different pens receive different feed; animals in the same pen receive the same feed. The weight gain of each animal is recorded.

  1. What is the unit of observation? Why?
  2. What is the unit of analysis? Why?
  3. Identify the between-individuals comparison.

Exercise 2.13 Consider this actual student RQ from the university where I work.

Among \(10\) Australian adults, does the time taken to read a passage of text change when different fonts are used?

Critique the RQ, and write a better RQ (if necessary).

Exercise 2.14 Consider this actual student RQ from the university where I work.

Of students that study at (a University), do males have a larger lung capacity than females?

Critique the RQ, and write a better RQ (if necessary).

Exercise 2.15 Consider this RQ, with an intervention:

For Japanese adults with a common cold, do people who take Vitamin C tablets daily have, on average, a shorter cold duration than people who do not take any Vitamin C tablets?

  1. What is the population?
  2. What is the comparsion?
  3. What is the outcome?
  4. What is the response variable?
  5. What is the explanatory variable?
  6. What type of RQ is this: estimation or decision-making?
  7. Is the RQ one-tailed or two-tailed?

Exercise 2.16 A research study was comparing the average size of Blue Gum eucalypt leaves in two areas of Queensland. A student takes \(40\) leaves from each of ten trees in Area A, and \(40\) leaves from each of ten trees in Area B.

Are the following statements true or false?

  1. The unit of analysis is the individual leaf.
  2. The unit of observation is the individual leaf.
  3. The unit of analysis is the tree.
What is the size of the sample in the study?

Exercise 2.17 A study (Prinz and Murray 2023) examined the strength needed to pull out nose-hairs. Fifty nose-hairs were pulled from one author's nose, and \(50\) nose hairs pulled from the other author's nose, and the average pull-out strengths for each man compared.

  1. What are the units of analysis and units of observation?
  2. What is the sample size in this study?

Exercise 2.18 A study (Huang et al. 2020) of environments on stress placed different people into one of three different virtual-reality (VR) environments: trees, grass or concrete. Stress levels were measured using 'skin conductance level' (SCL) for each individual, before and after exposure to the VR environment.

  1. Identify the between-individuals comparisons.
  2. Identify the within-individuals comparisons.
  3. Is the definition given for SCL (p. 2) a conceptual or operational definition?

SCLs are an unbiased measure of sympathetic activity via the electric impulses on the skin’s surface and sweat glands, which are innervated only by the sympathetic nervous system...

Exercise 2.19 Consider this two-tailed RQ (based on Tudor-Locke, Barreira, and Schuna Jr (2015)):

For American adults, is the average number of recorded steps per day the same for those using both a waist accelerometer, and a wrist accelerometer?

  1. Identify the population and the individuals.
  2. Identify the outcome.
  3. Identify the response and explanatory variables.
  4. Determine if the comparison is between- or within-individuals.

Exercise 2.20 Studies can incorporate many types of RQs. For example, a study (Thane, Bates, and Prentice 2004) of 'British young people aged \(4\)--\(18\)' asked and answered numerous RQs.

  1. What is the average zinc intake of the children?
  2. Does the average zinc intake meet recommended dietary guidelines?
  3. What is the strength of the association between plasma zinc and retinol concentrations?
  4. Is the average zinc intake the same for boys and girls?

For each RQ, classify these RQs as descriptive, relational, repeated-measures, or correlational RQs. Then, classify them as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.21 A study (Stern et al. 2021) examined the mean daily sodium excretion in Israeli adults. The study studied the relationship between daily sodium excretion and whether people had been diagnosed with diabetes or not. The study also explored the the relationship between the daily sodium excretion and the systolic blood pressure.

Classify the two RQs as descriptive, relational, repeated-measures, or correlational RQs. Then, classify them as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.22 A study studied the incidence of musculoskeletal disorders in Iranian bus drivers (Ghasemi and Pirzadeh 2019). The researchers introduced a program that aimed to provide relief for the drivers. Each bus driver was evaluated both before and after the intervention.

Classify the RQs as descriptive, relational, repeated-measures, or correlational RQs. Then, classify the RQ as estimation or decision-making RQs. Does the study have an invention?

Exercise 2.23 A study compares the wear on two brands of car tyres. Four tyres of Brand A are allocated to each of Cars 1--5, and four tyres of Brand B are allocated to each of Cars 6--10. After \(12\) months, the amount of wear is recorded on each tyre, and the two brands compared.

What are the units of analysis, and the units of observation?