Chapter 2 Errors in Surveying (and how to avoid them)
2.1 Survey Design Errors
Ponto (2015) describes why it is important to understand the errors which can occur in surveying:
“It is imperative for the consumer/reader of survey research to understand the potential for bias in survey research as well as the tested techniques for reducing bias, in order to draw appropriate conclusions about the information reported in this manner.” Ponto (2015)
| Types of error | Sources of error | Strategies to reduce error |
|---|---|---|
| Coverage error | Unknown or 0 chance of individuals in the population being included in the sample | Multimode design |
| Sampling error | Individuals included in the sample do not represent the characteristics of the population | Clearly identify population; diverse participant recruitment strategies; large, random sample |
| Measurement error | Questions / instrument do not accurately reflect the topic of interest; questionnaires / interviews do not evoke truthful answers | Valid, reliable instruments; pretest questions, user-friendly graphics, visual characteristics |
| Nonresponse error | Lack of response from all individuals in sample | user-friendly survey design; follow-up procedures for nonresponders |
| * Source: Ponto, 2015 |
2.1.1 Coverage errors
2.1.2 Sampling errors
Sampling errors, or errors caused wherein the individual responding to the survey does not represent the population as a whole, can __________.
Brenner and Bulgar-Medina (2018) => “mark all that apply” to gender / sexuality description
2.1.3 Measurement errors
Measurement errors are a broad group of errors that are centered around the questions themselves: their syntax, length, comprehension difficulty, type(multiple choice or open text?), _____. Researchers have been exploring the impact the questions themselves have on survey results since the 1950s (maybe longer), and it remains one of the most difficult to resolve errors.
In 1951, S.L. Payne produced a concise checklist of 100 considerations (???).
2.1.4 Nonresponse errors
Fowler et al. (2016) -> reducing nonresponse error in telephone surveys (case study)
References
Brenner, Philip S., and Justine Bulgar-Medina. 2018. “Testing Mark-All-That-Apply Measures of Sexual Orientation and Gender Identity.” Field Methods 30 (4): 357–70. https://doi.org/10.1177/1525822X18795872.
Fowler, Floyd J., Anthony M. Roman, Rumel Mahmood, and Carol A. Cosenza. 2016. “Reducing Nonresponse and Nonresponse Error in a Telephone Survey: An Informative Case Study.” Journal of Survey Statistics and Methodology 4 (2): 246–62. https://doi.org/10.1093/jssam/smw004.
Ponto, Julie. 2015. “Understanding and Evaluating Survey Research.” Journal of the Advanced Practitioner in Oncology 6 (2): 168–71.