37 Reading research

So far, you have learnt to ask a RQ, identify different ways of obtaining data, design the study, collect the data describe the data, summarise data graphically and numerically, understand the tools of inference, to form confidence intervals, and to perform hypothesis tests.

In this chapter, you will learn to read about the research of others. You will learn to:

  • read and understand research.

37.1 Introduction

Science requires reading the research of others. Research is usually communicated in journal articles (also called papers), or sometimes in presentations (conferences; seminars). Milllions of journal articles are available (many online), and this book references many articles.

At some time during your university studies, you will need to read articles: so you know why your discipline does things as it does, the evidence for doing so, and open questions in your discipline. Understanding the language of research is important for understanding these articles.

However, reading a research article can be hard work... A good place to start is to read the Abstract (sometimes called a Summary, or Overview): a useful overview of the whole paper (without details).

To understand a paper, the six steps of the research process can be used as a guide::

  1. Ask the question: What research question is the paper answering? Are inclusion and/or exclusion criteria given?
  2. Design the study: How did the authors design the study? Is the study designed to maximize internal and external validity? What are the design limitations?
  3. Collect the data: How did the authors collect the data? Could the study be approximately repeated if needed?
  4. Describe and summarise the data: Is the data summary appropriate, complete and clear?
  5. Analyse the data: Is the analysis appropriate, accurate, valid and clear?
  6. Report the results: Are the results accurately, appropriately and well reported? What is the answer to the RQ? What other questions have emerged?

In the examples that follow, some extracts from articles will be studied.

37.2 Example 1: Reading research

A study532 explored the impact of adding herbs and spices to the consumption of vegetables by adolescent school children. Part of the Abstract states (slightly edited for brevity):

Purpose: We evaluated whether new vegetable recipes using herbs and spices would increase preference for vegetables served to adolescents at this school.

Methods: To evaluate recipe acceptance, we assessed liking (100 mm visual analog scales) among students (\(n =\) 96--110; aged 14--18 years) for 8 plain (oil and salt) and 8 seasoned vegetables. Liking ratings between plain and seasoned vegetables were compared with paired \(t\)-tests...

Results: Students reported higher liking for several seasoned recipes compared to plain: broccoli (\(P = 0.02\)), vegetable dip (\(P < 0.0001\)), black beans and corn (\(P < 0.001\)) and cauliflower (\(P < 0.0001\)).

Conclusions: Common herbs and spices improved liking for several school lunch vegetables compared to plain varieties among rural high school students...

--- Fritts et al.533, p. 125

Later we read this (again, slightly edited):

This is a cross-sectional study assessing preference for plain and seasoned vegetables in a population of middle/high school students (aged 14--18 years) attending a rural Pennsylvania public school.

--- Fritts et al.534, p. 126

Even using this (small amount) of information, much can be learnt about the study. For example:

  1. Ask the question: The POCI elements are:

    • Population: 'middle/high school students (aged 14--18 years) attending a rural Pennsylvania public school'
    • Outcome: The mean difference in taste ratings between plain and seasoned vegetables. The taste ratings are given using a '100 mm visual analog scale'.
    • Comparison: There is no comparison: Every member of the population is treated the same way. A comparison exists if different subsets of the population are treated differently (for example, one group of students is given plain vegetables, and a different group is given seasoned vegetables).
    • Intervention: No; there is no comparison, so there is no comparison to be allocated.
  2. Design the study: Since this RQ is descriptive, the study is * descriptive. The participants were probably not blinded*, since the presence of seasoning was probably obvious.

  3. Collect the data: No details are given about the data collection.

  4. Describe and summarise the data: The Abstract gives no summary data (since eight vegetables were studied, this would have consumed too much space I guess).

  5. Analyse the data: The data were analysed using paired \(t\)-tests, one for each different vegetable used. (Each subject gave two ratings for each vegetable: one for plain vegetables and one for seasoned vegetables),

  6. Report the results: Evidence exists of a mean difference (that students preferred the seasoned vegetables) in many cases, but not all (the Abstract states that eight vegetables were used, with statistically significant differences for five).

From this information, the RQ is something like:

For middle/high school students (aged 14--18 years) attending a rural Pennsylvania public school, is there a mean difference in taste ratings (measured on a '100 mm visual analog scale') between plain and seasoned vegetables?

For more details, the whole paper could be read.

37.3 Example 2: Reading research

Consider this Abstract:535

Objective To determine whether the author's 20.9 lb (9.5 kg) carbon frame bicycle reduced commuting time compared with his 29.75 lb (13.5 kg) steel frame bicycle.

Design Randomised trial.

Setting Sheffield and Chesterfield, United Kingdom, between mid-January 2010 and mid-July 2010.

Participants One consultant in anaesthesia and intensive care.

Main outcome measure Total time to complete the 27 mile (43.5 kilometre) journey from Sheffield to Chesterfield Royal Hospital and back.

Results The total distance travelled on the steel frame bicycle during the study period was 809 miles (1302 km) and on the carbon frame bicycle was 711 miles (1144 km). The difference in the mean journey time between the steel and carbon bicycles was 00:00:32 (hr:min:sec; 95% CI -00:03:34 to 00:02:30; \(P=0.72\)).

Conclusions A lighter bicycle did not lead to a detectable difference in commuting time. Cyclists may find it more cost effective to reduce their own weight rather than to purchase a lighter bicycle.

--- Groves536, p. 341

Based on this Abstract, again we can learn many things about the study.

  1. Ask the question: The POCI elements are:

    • Population: The trips by this rider, on his bikes, on his route to work. This is not easy to identify, but notice that there are many examples of this rider, on his bikes, on his route. For example, there are not many examples of different bikes, different riders, or different routes.
    • Outcome: 'Total time to complete the 27 mile (43.5 kilometre) journey'.
    • Comparison: Between the steel-frame and carbon-frame bicycles.
    • Intervention: Yes, because the elements of the population (the different commutes) can be randomly allocated to be taken with the steel- or carbon-frame bikes.
  2. Design the study: The study is 'randomised controlled trial', a type of experimental study. Random allocation has been used.

  3. Collect the data: The Abstract gives no information.

  4. Describe and summarise the data: The Abstract gives no summary data for each bike, but summarises the difference between the means: 32 seconds (95% CI between -3:34 and 2:30 minutes, but which bike produces the faster mean time is not stated).

  5. Analyse the data: Though not stated, probably a two-sample \(t\)-test.

  6. Report the results: 'A lighter bicycle did not lead to a detectable difference in commuting time': There is no evidence that the carbon-frame bicycle reduced the commmuting time (for this rider, on his route to work, with his bikes...). In any case, the difference between the two mean commuting times is 32 seconds... over a 43.5 kilometre journey: Hardly of any practical importance (Sect. 29.8)!

The RQ may be:

For trips made by one cyclist (on his bikes, on his route to work), is the mean time to complete the 43.5 kilometre the same for the steel-frame and carbon-frame bicycles?

This is a poor RQ: it is not relevant or interesting (Sect. 2.6) to anyone except this single rider: The results are relevant to one person in the entire world...

Another thing to observe: The RQ is one-tailed (does the carbon frame bicycle reduce commuting time), but the conclusion gives a two-tailed \(P\)-value. (This may not be obvious, but a one-tailed \(P\)-value cannot be larger than 0.5.)

This is a strange study... However, it appeared in a Christmas edition of BMJ, which contains more 'light-hearted' articles:

While the subject matter may be more light-hearted, research papers in the Christmas issue adhere to the same high standards of novelty, methodological rigour, reporting transparency, and readability as apply in the regular issue.

--- From https://www.bmj.com/about-bmj/resources-authors/article-types/christmas-issue

37.4 Exercises

Selected answers are available in Sect. D.34.

Exercise 37.1 A research article537 examined the accuracy of step counts recorded on iPhones. The paper records this information about the selection of participants:

Participants were recruited through word of mouth and posters displayed around the [researcher's] university. Participants were eligible if they were ambulatory, \(\ge 18\) years of age, and owned an iPhone 6 [...] or newer model.

Although 33 participants were selected, the authors note some parts of the study used a smaller sample size because:

... one [subject] lost their phone during the observation period, [and] the other opted out of the [...] test due to personal circumstances.

The paper notes that previous studies have been able to:

[...] demonstrate the accuracy of the iPhone pedometer function in laboratory test conditions. However, no studies have attempted to evaluate evidence [...] in the field.

  1. What is the issue that the authors raise with previous studies?
  2. Why did the authors discuss the changes in sample size for some parts of the study?
  3. How would you describe the sampling method?
  4. What would you call the information about given about the subjects needing to be ambulatory and 18 years of age or over?
  5. Among many other things, the researchers compared the mean diference between the number of step counts recorded by manually counting steps and the iPhone-recorded number of steps. What type of test would be appropriate?
  6. While walking at 2.5 km/h, the above test produced a \(P\)-value of 0.006.
    What does this mean?
  7. The sample size for the part of the study mentioned above was \(n=32\). Do you think the test will be statistically valid?

Exercise 37.2 One study of hearing loss among Iranian students538 used a cross-sectional study to explore the relationship between hearing loss and headphone use.

The article states that

... 890 students were randomly selected from five schools at QUMS (Medicine, Dentistry, Nursing and Midwifery, Public Health, and Paramedical Sciences schools) using a proportional cluster sampling method...

The participants completed a hearing test and completed a Hearing Loss Questionnaire (values are between 17 and 34: higher scores indicating more severe hearing loss).

  1. What is the population?
  2. Critique the sampling method: What is the implication for interpreting the results of the study?
  3. Some of the results are presented in Table 37.1. What statistical test do you think was used to compare the scores for males and females?
  4. What are the hypotheses being tested about 'Frequency of use'?
  5. Form an approximate 95% CI for the mean hearing loss score for students who use earphones.
  6. What information is needed to be able to form an approximate 95% CI for the difference between the hearing loss scores for females and males?
TABLE 37.1: The Hearing Loss Questionnaire scores for various demographic variables
Criterion Levels Sample size Mean Std. dev P-value
Sex Female 543 19.37 2.91 0.009
Male 302 19.99 3.51
Frequency of use 0, 1 times/day 194 19.2 2.87 0.001
2 to 3 times/day 319 19.6 2.66
More than 3 times/day 278 20.2 3.54
Earphone use Yes 745 19.8 3.08 < 0.001
No 100 19 1.71

Exercise 37.3 The Abstract from a large study is given below:

OBJECTIVE: This study aims to elucidate any existing link between energy-containing liquids, as consumed in various forms within the diet, and the effect they may have on body weight or other diseases [...]

METHODS: A self-administered online survey was conducted in 2496 participants from different countries, in six languages (Spanish, English, Chinese, French, German and Portuguese). Questions referred to their soft drink and water consumption habits, physical exercise performed, presence or absence of certain diseases and medication.

RESULTS: There is statistically significant difference (\(p < 0.001\)) in BMI and consumption of cola per week: those who consumed 0--3 cans a week have a lower BMI than those who consume >7 cans of cola a week [...] There is greater presence of obesity (\(p < 0.001\)), gastritis (p < 0.001), constipation (\(p < 0.001\)) and mental illness (\(p = 0.003\)) among people who drink cola soft drinks.

CONCLUSION: Removal of energy-containing beverages from our diet may be an appropriate public health message to support those interested in preventing weight gain as well as other diseases.

Ismael San Mauro Martín et al.539, p. 1

Evaluate the study using the six steps of research discussed in this book.