Discussion: Interpretation (20)
The items from STROBE state that you should report:
- Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence
Some key items to consider adding:
- Give an interpretation of results in terms of a priori biological plausibility
Explanation
The heart of the discussion section is the interpretation of a study’s results. Over-interpretation is common and human: even when we try hard to give an objective assessment, reviewers often rightly point out that we went too far in some respects. When interpreting results, authors should consider the nature of the study on the discovery to verification continuum and potential sources of bias, including loss to follow-up and non-participation (see also items 9, 12 and 19). Due consideration should be given to confounding (item 16a), the results of relevant sensitivity analyses, and to the issue of multiplicity and subgroup analyses (item 17). Authors should also consider residual confounding due to unmeasured variables or imprecise measurement of confounders. For example, socioeconomic status (SES) is associated with many health outcomesand often differs between groups being compared. Variables used to measure SES (income, education, or occupation) are surrogates for other undefined and unmeasured exposures, and the true confounder will by definition be measured with error. (Kaufman et al., 1997) Authors should address the real range of uncertainty in estimates, which is larger than the statistical uncertainty reflected in confidence intervals. The latter do not take into account other uncertainties that arise from a study’s design, implementation, and methods of measurement. (Greenland, 1990)
To guide thinking and conclusions about causality, some may find criteria proposed by Bradford Hill in 1965 helpful. (Hill, 1965) How strong is the association with the exposure? Did it precede the onset of disease? Is the association consistently observed in different studies and settings? Is there supporting evidence from experimental studies, including laboratory and animal studies? How specific is the exposure’s putative effect, and is there a dose-response relationship? Is the association biologically plausible? These criteria should not, however, be applied mechanically. For example, some have argued that relative risks below 2 or 3 should be ignored.(Tabues, 1995; Temple, 1999) This is a reversal of the point by Cornfield et al about the strength of large relative risks (see item 12b). (Cornfield et al., 1959) Although a causal effect is more likely with a relative risk of 9, it does not follow that one below 3 is necessarily spurious. For instance, the small increase in the risk of childhood leukemia after intrauterine irradiation is credible because it concerns an adverse effect of a medical procedure for which no alternative explanations are obvious.(Greenberg & Shuster, 1985) Moreover, the carcinogenic effects of radiation are well established. The doubling in the risk of ovarian cancer associated with eating 2 to 4 eggs per week is not immediately credible, since dietary habits are associated with a large number of lifestyle factors as well as SES. (Kushi et al., 1999) In contrast, the credibility of much debated epidemiologic findings of a difference in thrombosis risk between different types of oral contraceptives was greatly enhanced by the differences in coagulation found in a randomized cross-over trial. (Kemmeren et al., 2004) A discussion of the existing external evidence, from different types of studies, should always be included, but may be particularly important for studies reporting small increases in risk. Further, authors should put their results in context with similar studies and explain how the new study affects the existing body of evidence, ideally by referring to a systematic review.
Example
- “Any explanation for an association between death from myocardial infarction and use of second generation oral contraceptives must be conjectural. There is no published evidence to suggest a direct biologic mechanism, and there are no other epidemiologic studies with relevant results. (. . .) The increase in absolute risk is very small and probably applies predominantly to smokers. Due to the lack of corroborative evidence, and because the analysis is based on relatively small numbers, more evidence on the subject is needed. We would not recommend any change in prescribing practice on the strength of these results.” (Dunn et al., 2001; Vandenbroucke et al., 2007)
Field-specific guidance
Seroepidemiologic studies for influenza (Horby et al., 2017)
- Discuss the interpretation of the results in the context of known or potential cross-reactivity
Nutritional data (Lachat et al., 2016)
- Report the nutritional relevance of the findings, given the complexity of diet or nutrition as an exposure
References
Cornfield, J., Haenszel, W., Hammond, E. C., Lilienfeld, A. M., Shimkin, M. B., & Wynder, E. L. (1959). Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions. JNCI: Journal of the National Cancer Institute, 22(1), 173–203. https://doi.org/10.1093/jnci/22.1.173
Dunn, N. R., Arscott, A., & Thorogood, M. (2001). The relationship between use of oral contraceptives and myocardial infarction in young women with fatal outcome, compared to those who survive: Results from the MICA case-control study. Contraception, 63(2), 65–69. https://doi.org/10.1016/S0010-7824(01)00172-X
Greenberg, R., & Shuster, J. (1985). Epidemiology of cancer in children. Epidemiologic Reviews, 7, 22–48. https://doi.org/10.1093/oxfordjournals.epirev.a036284
Greenland, S. (1990). Randomization, statistics, and causal inference. In Epidemiology 1.
Hill, B. (1965). The environment and disease: Association or causation? Proc R Soc Med, 58, 295–300. https://journals.sagepub.com/doi/full/10.1177/0141076814562718
Horby, P. W., Laurie, K. L., Cowling, B. J., Engelhardt, O. G., Sturm-Ramirez, K., Sanchez, J. L., Katz, J. M., Uyeki, T. M., Wood, J., Van Kerkhove, M. D., & the CONSISE Steering Committee. (2017). CONSISE statement on the reporting of Seroepidemiologic Studies for influenza (ROSES-I statement): An extension of the STROBE statement. Influenza and Other Respiratory Viruses, 11(1), 2–14. https://doi.org/10.1111/irv.12411
Kaufman, J. S., Cooper, R. S., & McGee, D. L. (1997). Socioeconomic status and health in blacks and whites: The problem of residual confounding and the resiliency of race. Epidemiology, 8(6), 621–628. http://www.jstor.org/stable/3702653
Kemmeren, J. M., Algra, A., Meijers, J. C. M., Tans, G., Bouma, B. N., Curvers, J., Rosing, J., & Grobbee, D. E. (2004). Effect of second- and third-generation oral contraceptives on the protein C system in the absence or presence of the factor VLeiden mutation: A randomized trial. Blood, 103(3), 927–933. https://doi.org/10.1182/blood-2003-04-1285
Kushi, L. H., Mink, P. J., Folsom, A. R., Anderson, K. E., Zheng, W., Lazovich, D., & Sellers, T. A. (1999). Prospective Study of Diet and Ovarian Cancer. American Journal of Epidemiology, 149(1), 21–31. https://doi.org/10.1093/oxfordjournals.aje.a009723
Lachat, C., Hawwash, D., Ocké, M. C., Berg, C., Forsum, E., Hörnell, A., Larsson, C., Sonestedt, E., Wirfält, E., Åkesson, A., Kolsteren, P., Byrnes, G., De Keyzer, W., Van Camp, J., Cade, J. E., Slimani, N., Cevallos, M., Egger, M., & Huybrechts, I. (2016). Strengthening the Reporting of Observational Studies in Epidemiology—Nutritional Epidemiology (STROBE-nut): An Extension of the STROBE Statement. PLOS Medicine, 13(6), e1002036. https://doi.org/10.1371/journal.pmed.1002036
Tabues, G. (1995). Epidemiology faces its limits. Science, 269, 164–169. https://search.proquest.com/openview/a57c6f7c863db0787bf3ee25df2cdeae/1?pq-origsite=gscholar&cbl=1256
Temple, R. (1999). Meta-analysis and Epidemiologic Studies in Drug Development and Postmarketing Surveillance. JAMA, 281(9), 841–844. https://doi.org/10.1001/jama.281.9.841
Vandenbroucke, J. P., Elm, E. von, Altman, D. G., Gotzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007). Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration. Epidemiology, 18(6), 805–835. https://doi.org/10.1097/EDE.0b013e3181577511