Week 1: Surveys and experiments
Survey data is one of the most regularly used type of data—certainly when we’re interested in measuring individual-level attitudes and preferences. But surveys are also how we measure unemployment, population, and migration etc. That is, we can use them to measure aggregate population-level characteristics too. And what we’ll see in the readings for this week is that surveys can also be combined with experimental strategies for estimating the causal effect of given ``treatments" of interest. We will compare these to other experimental strategies for measuring real-world behaviours.
We will have the opportunity to discuss surveying techniques, more recent innovations in so-called “survey experiments” (Nugent 2020; Bush et al. 2016; Nugent, Masoud, and Jamal 2018), the potentials and pitfalls of survey techniques designed to overcome social desirability bias (Truex and Tavana 2019; Brooke 2017; Nillesen et al. 2021), and experimental setups designed to measure real-world behavior (Mousa 2020).
For each of these empirical papers, I have selected some more general readings, and further case studies, pertinent to the issues raised in each. The papers by Krumpal (2013) and Ksiazkiewicz and Hedrick (2013) provide general overviews of issues of social desirability bias and so-called “implicit attitudes” techniques in social cognition research. The article by Schuman and Johnson (1976) provides an early overview of the so-called attitude-behaviour gap in survey research, while the Barabas and Jerit (2010) article investigates the external (i.e., real-world relevance) of lab-based experiments. The book by Kuran (1995) is a classic of social science research and should be consulted in part, if not in full by anyone with an interest in studying attitudes generally. The World Politics article by Kuran (1991) provides an accessible application of the theory of preference falsification to contexts of political revolution.
The aim of the practical exercise is to help you understand, analyse, and reproduce aggregate descriptive statistics. Here, we will be using data from a recent article in the Journal of Politics by Truex and Tavana (2019) that investigates levels of support for the authoritarian government of Abdel Fattah El-Sisi in Egypt. The exercise will show you how to replicate their results in R and you will also be set some coding tasks to complete yourselves.
Questions to consider in the seminar: Can we rely on individuals to give honest answers? Can we accurately measure complex concepts in surveys? What criteria might we use to ensure the validity of survey instruments? Can we measure opinions without directly asking for them? How might we overcome response biases in survey research? Does experimental evidence tell us something meaningful about real-world social phenomena?
Required reading:
General reading:
- Barabas and Jerit (2010)
- Ksiazkiewicz and Hedrick (2013)
- Kuran (1995)
- Kuran (1991)
- Krumpal (2013)
- Schuman and Johnson (1976)
Additional case studies reading:
References
Barabas, Jason, and Jennifer Jerit. 2010. “Are Survey Experiments Externally Valid?” American Political Science Review 104 (2): 226–42. https://doi.org/10.1017/S0003055410000092.
Brooke, Steven. 2017. “Sectarianism and Social Conformity: Evidence from Egypt.” Political Research Quarterly 70 (4): 848–60. https://doi.org/10.1177/1065912917717641.
Bush, Sarah Sunn, Aaron Erlich, Lauren Prather, and Yael Zeira. 2016. “The Effects of Authoritarian Iconography: An Experimental Test.” Comparative Political Studies 49 (13): 1704–38. https://doi.org/10.1177/0010414016633228.
Krumpal, Ivar. 2013. “Determinants of Social Desirability Bias in Sensitive Surveys: A Literature Review.” Qual Quant 47: 2025–47. https://doi.org/10.1007/s11135-011-9640-9.
Ksiazkiewicz, Aleksander, and James Hedrick. 2013. “Implicit Attitudes in Political Science Research.” PS - Political Science and Politics 46 (3): 525–31. https://doi.org/10.1017/S1049096513000632.
Kuran, Timur. 1995. Private Truths, Public Lies: The Social Consequences of Preference Falsification. Cambridge: Harvard University Press.
Kuran, Timur. 1991. “Now Out of Never: The Element of Surprise in the East European Revolution of 1989.” World Politics 44 (1): 7–48.
Mousa, Salma. 2020. “Building Social Cohesion Between Christians and Muslims Through Soccer in Post-ISIS Iraq.” Science 369 (6505): 866–70. https://doi.org/10.1126/science.abb3153.
Nillesen, Eleonora, Michael Grimm, Micheline Goedhuys, Ann-Kristin Reitmann, and Aline Meysonnat. 2021. “On the Malleability of Gender Attitudes: Evidence from Implicit and Explicit Measures in Tunisia.” World Development 138 (February): 105263. https://doi.org/10.1016/j.worlddev.2020.105263.
Nugent, Elizabeth, Tarek Masoud, and Amaney A. Jamal. 2018. “Arab Responses to Western Hegemony: Experimental Evidence from Egypt.” Journal of Conflict Resolution 62 (2): 254–88. https://doi.org/10.1177/0022002716648738.
Nugent, Elizabeth R. 2020. “The Psychology of Repression and Polarization.” World Politics 72 (2): 291–334. https://doi.org/10.1017/S0043887120000015.
Schuman, H, and M P Johnson. 1976. “Attitudes and Behavior.” Annual Review of Sociology 2: 161–207. https://doi.org/10.1146/annurev.so.02.080176.001113.
Truex, Rory, and Daniel L. Tavana. 2019. “Implicit Attitudes Toward an Authoritarian Regime.” The Journal of Politics 81 (3): 1014–27. https://doi.org/10.1086/703209.