Threat-Inducing Violent Events Exacerbate Social Desirability Bias in Survey Responses

The forthcoming article “Threat-Inducing Violent Events Exacerbate Social Desirability Bias in Survey Responses” by Shane P. Singh and Jaroslav Tir is summarized by the author(s) below. 

Feeling pressure to answer in a socially desirable manner, survey respondents sometimes provide untruthful answers to the questions asked of them. We identify violent events as a new source of social desirability bias. We theorize that, because violent events present a threat to society, individuals feel pressured to report engaging in behaviors that are perceived as being supportive of it. Our findings have important substantive and methodological implications for researchers who rely on survey questions known to be susceptible to social desirability bias—particularly when these are asked in the wake of violent conflict.  

To detect social desirability bias, we focus on responses to questions about past voter turnout, which are known to be influenced by social desirability. If our theory is correct, the likelihood that individuals will report having voted when they did not should be higher when they have been exposed to violent events. To marshal evidence for this observable implication, we conduct three separate studies.  

The first leverages the timing of survey fieldwork in the Comparative Study of Electoral Systems across 103 elections. We compare individuals whose post-election survey interviews were preceded by a post-election fatal terrorist attack to those whose were not. We find that being surveyed after an attack is associated with about a seven percentage point increase in the probability of reporting turnout. Because we only consider post-election fatal terror attacks, it is impossible for the attacks to have genuinely influenced electoral behavior. Thus, differences in recalled participation are attributable to turnout overreporting. 

The second study leverages the 2004 European Social Survey in the Netherlands, fieldwork for which was interrupted by the murder of Theo van Gogh, a prominent critic of Islam who was violently slain on an Amsterdam street. We find that the van Gogh terror attack boosted reported turnout among Dutch respondents by nearly eight percentage points when comparing individuals who responded to the survey after as opposed to before the murder. Because every individual was interviewed after the election in question, this increase is attributable to turnout overreporting.  

In the third study, we designed and implemented a randomized survey experiment in India that leverages a deadly 2019 terror attack on Indian national police. We find that respondents who were shown a vignette about the attack were about ten percentage points more likely to report having voted in the most recent national election than those in who were asked to read an innocuous vignette. 

To avoid making erroneous inferences, researchers should pay careful attention to the timing of surveys vis-à-vis potential violent events that take place during fieldwork. Our results show that respondents are prone to providing society-supporting responses, even when these are misrepresentations, in the wake of violent events. Our findings also suggest that standard social desirability mitigating approaches (e.g., question wording that excuses nonvoting) cannot alone correct for this bias.  

About the Author(s): Shane P. Singh is Professor, Department of International Affairs, School of Public and International Affairs at University of Georgia and Jaroslav Tir is Professor, Department of Political Science at University of Colorado Boulder. Their research “Threat-Inducing Violent Events Exacerbate Social Desirability Bias in Survey Responses” is now available in Early View and will appear in a forthcoming issue of the American Journal of Political Science.