Introduction
Well-functioning democracies entail political participation by ordinary citizens (Verba and Nie, Reference Verba and Nie1972; Van Deth, Reference Van Deth2001; Dahl, Reference Dahl2005). Recently, there has been a growing concern about how the harshness in political discussions, especially on social media, may affect ordinary citizens who witness such discussions. For example, a growing strand of research has investigated whether incivility in political discussion, that is, discussions where the norms of interpersonal interaction are violated (Mutz, Reference Mutz2015), affects citizens’ political engagement. So far, the empirical evidence does not consistently support a demobilization hypothesis (Van’t Riet and Van Stekelenburg, Reference Van’t Riet and Van Stekelenburg2022).
However, incivility is just one expression of political discussions being harsh, and it is arguably a comparatively mild one. While eye-rolling, vulgarities, and other expressions of incivility might be off-putting to some citizens, recent studies have documented the prevalence of much more severe kinds of online political behavior, including threats, racism, or sexism, particularly against politicians on social media platforms (James et al., Reference James, Sukhwal, Farnham, Evans, Barrie, Taylor and Wilson2016; Rheault et al., Reference Rheault, Rayment and Musulan2019; Gorrell et al., Reference Gorrell, Bakir, Roberts, Greenwood and Bontcheva2020). As recent studies suggest, online abuse of such a caliber can have negative effects on the involved politicians’ mental health and well-being and may also dampen their participation in public discussions and desire to continue their political career (James et al., Reference James, Sukhwal, Farnham, Evans, Barrie, Taylor and Wilson2016; Theocharis et al., Reference Theocharis, Barberá, Fazekas, Adrian Popa and Parnet2016; Tromble, Reference Tromble2018; Collignon and Rüdig, Reference Tromble2020). Similarly, online abuse of politicians could also have negative effects on bystanding citizens’ prospective political participation. In this article, we therefore investigate whether online abuse directed at politicians on social media affects the prospective political participation of ordinary citizens who witness the abuse.
Specifically, we test the hypothesis that citizens’ prospective political participation decreases when they are exposed to online abuse of politicians. We measure citizens’ willingness to participate in politics both by their willingness to engage in online political activities, their willingness to engage in offline political activities, and their willingness to run for political office.Footnote 1 As examples of online abuse, our experimental design includes real cases from Facebook and Twitter (now X) of insults, sexism, and violent threats directed at politicians. We also test two hypotheses about the role of gender and conflict avoidance to investigate whether average treatments effects conceal effects among some subgroups: women may be particularly prone to withdrawing from active engagement in politics, as conflict aversion is correlated with gender (Mutz, Reference Mutz2015; Schneider et al., Reference Schneider, Holman, Diekman and McAndrew2016). Such heterogenous effects are important to investigate, as they could exacerbate existing gender inequalities in politics.
Contrary to expectations, our preregistered survey experiment in Denmark shows that exposure to online abuse of politicians does not have any negative effects on citizens’ prospective political participation. This holds irrespective of whether we consider overall participation or distinguish between different kinds of political participation. The results are also robust when distinguishing between men and women and between people with low and high levels of conflict avoidance. If anything, our results tentatively suggest that exposure to online abuse of politicians may in some cases increase citizens’ prospective political participation.
Online abuse of politicians
Abusive messages have been defined as messages “directed at a specific person with the intent to cause harm or distress” (Ward and McLoughlin, Reference Ward and McLoughlin2020). While this definition is open to subjective interpretation, surveys show that politicians and citizens alike generally find messages directed at politicians to be abusive when they contain rude and personal insults, overt sexist comments and – particularly – threats of violence (Rasmussen, Reference Rasmussen2022; Pedersen et al., Reference Pedersen, Petersen and Thau2024a; Petersen et al., Reference Petersen, Tue Pedersen and Thau2024).
Content analyses of social media messages directed at politicians have shown a relatively high prevalence of such types of abuse (Theocharis et al., Reference Theocharis, Barberá, Fazekas, Adrian Popa and Parnet2016; Rheault et al., Reference Rheault, Rayment and Musulan2019; Farrell et al. Reference Farrell, Gorrell and Bontcheva2020; Kosmidis and Theocharis, Reference Kosmidis and Theocharis2020; Unkel and Kümpel, Reference Unkel and Sophie Kümpel2022). Consistent with these findings from content analyses, surveys conducted among politicians also indicate that a large proportion of politicians have been insulted, threatened, or otherwise abused on social media (James et al., Reference James, Sukhwal, Farnham, Evans, Barrie, Taylor and Wilson2016; Collignon and Rüdig, Reference Collignon and Rüdig2020; Herrick and Thomas, Reference Herrick and Thomas2022; Pedersen et al., Reference Pedersen, Petersen and Thau2024a).
While social media companies continually remove abuse and other types of inappropriate content from their platforms (Douek, Reference Douek2022), much of the abuse directed at politicians remains on social media sites at least for some time, as evidenced by the amount of abuse found in content analyses. Previous studies have found incivility in 10–23% of social media messages and comments to politicians (Rheault et al., Reference Rheault, Rayment and Musulan2019; Theocharis et al., Reference Theocharis, Barberá, Fazekas and Adrian Popa2020; Unkel and Kümpel, Reference Unkel and Sophie Kümpel2022) and outright abuse in 2–5% of the comments and messages (Theocharis et al., Reference Theocharis, Barberá, Fazekas, Adrian Popa and Parnet2016, Reference Theocharis, Barberá, Fazekas and Adrian Popa2020; Farrell et al., Reference Farrell, Gorrell and Bontcheva2020; Gorrell et al., Reference Gorrell, Bakir, Roberts, Greenwood and Bontcheva2020). Further, a survey among Danish citizens found that 88% of Facebook users and 80% of Twitter users had seen hateful comments directed at someone else on these social media sites, and 57% responded that the hateful comments they had seen were about political beliefs (Jaeger et al., Reference Jaeger, Simon, Karg, Hebbelstrup, Rasmussen, Pradella, Rasmussen, Lindekilde and Petersen2022). Thus, while we do not have precise estimates of how often ordinary citizens observe online abuse specifically directed at politicians, it seems reasonable to assume that a substantial number of citizens observe some of the abuse hurled at politicians on social media.
The participatory consequences of online abuse
In surveys, politicians report that online abuse has made them anxious, concerned for their safety and the safety of those around them, and generally affected their personal well-being (James et al., Reference James, Sukhwal, Farnham, Evans, Barrie, Taylor and Wilson2016; Collignon and Rüdig, Reference Collignon and Rüdig2020; Pedersen et al., Reference Pedersen, Petersen and Thau2024a). Politicians also report that online abuse can affect how they serve their democratic functions, for example, the extent to which they partake in public debates, including discussions on social media, and the political issues they engage with. For some, at least according to self-reports, it may even affect their willingness to run for office in future elections (Pedersen et al., Reference Pedersen, Petersen and Thau2024a).
Importantly, online abuse of politicians could also have broader democratic consequences by affecting the political participation of ordinary citizens. Political participation requires not only resources but also motivation (Brady et al., Reference Brady, Verba, Lehman and Boston1995; Harder and Krosnick, Reference Harder and Krosnick2008). When citizens observe politicians being abused on social media, their political motivation may suffer for at least two reasons. First, citizens might become worried about participating in politics, fearing that they themselves risk being targeted by abuse. Second, citizens who are bystanders to such abuse might simply be turned off by politics if they get the impression that the domain of politics is generally unpleasant. While this demobilization hypothesis is untested in the literature on abuse of politicians, it has received considerable attention in the related literature on political incivility. Incivility has been defined as “violation of norms of interpersonal interaction” (Mutz, Reference Mutz2015) or simply “impoliteness” (Masullo et al., Reference Masullo, Lu and Fadnis2021). As such, incivility is a broader concept than online abuse and includes, for example, name-calling, coarse language, and non-verbal behavior such as eye-rolling (Otto et al., Reference Otto, Lecheler and Schuck2020). So far, the results from this literature are inconclusive. While some studies have found a demobilizing effect of incivility (Han and Brazeal, Reference Han and Brazeal2015), other studies have found that incivility can make people more emotionally engaged in politics (Borah, Reference Borah2014).Footnote 2 Recently, a meta-analysis of 19 studies concluded that the overall effect of incivility on political participation is small and statistically insignificant (Van’t Riet and Van Stekelenburg, Reference Van’t Riet and Van Stekelenburg2022).
However, there are several reasons as to why the lack of a consistent demobilizing effect in studies of incivility should not be used to infer much about the demobilizing effect of online abuse. First, from a methodological standpoint, one should be careful not to infer confidently from individual studies on incivility, as these have generally relied on experiments with relatively small sample size, limiting the statistical power to detect effects. For example, the 19 studies included in the meta-analysis by Van’t Riet and Stekelenburg (Reference Van’t Riet and Van Stekelenburg2022) had average sample sizes of just 277 participants (in contrast, we include 2,000 participants in our experiment). While the meta-analysis mitigates the problem with small sample sizes, the findings still rest on the aggregation of studies investigating very different types of incivility across distinct contexts.
Second, from a theoretical standpoint, online abuse can be of a character that is far more aggressive and threatening than behavior that is merely uncivil. With few exceptions (Hutchens et al., Reference Hutchens, Silva, Hmielowski and Cicchirillo2019), the incivility included in experiments is often in the form of nonthreatening use of vulgarities such as “It makes me want to puke” (Kluck and Krämer, Reference Kluck and Krämer2022) and derogatory slurs such as “assholes” and “idiots” (Otto et al., Reference Otto, Lecheler and Schuck2020; Kluck and Krämer Reference Kluck and Krämer2022). Furthermore, the incivility is often not directed at specific individuals or groups and can also be part of longer messages and comments containing substantive arguments regarding policy positions. In contrast, the online abuse whose effects we investigate is directed at a specific individual and includes statements such as death threats, which go far beyond mere incivility (see Appendix A for specific examples of online abuse used in our experiment). For these reasons, online abuse may have a stronger demobilizing effect than at least some types of incivility. Thus, we test the following hypothesis:
Hypothesis 1: Citizens’ prospective political participation is decreased when exposed to online abuse of politicians.
Gender and conflict avoidance
The effects of being exposed to online abuse of politicians could potentially be stronger among women than men. Previous studies among politicians and ordinary citizens have clearly shown that women tend to be more averse to abusive messages and ostensibly more affected by such messages (Collignon and Rüdig, Reference Collignon and Rüdig2020; Pedersen et al., Reference Pedersen, Petersen and Thau2024a).
An explanation for differential effects among men and women can be that they differ substantially in their average level of conflict avoidance. Men are generally socialized in ways that emphasize assertiveness and argumentation. As a consequence, men are less averse to conflicts, on average, while women are more likely to find arguments unpleasant (Wolak, Reference Wolak2022). Notably, men and women differ markedly on aversion to public conflicts, with women being substantially more averse to public, rather than private, conflicts (Mutz and Reeves, Reference Mutz and Reeves2005, 144). These gender differences are important to examine as women are already underrepresented in politics. If women are demobilized more than men by online abuse of politicians, such abuse may potentially exacerbate the already existing gender inequalities in politics. We, therefore, test the following hypotheses:
Hypothesis 2: Exposure to online abuse of politicians particularly decreases prospective political participation among women.
Hypothesis 3: Exposure to online abuse of politicians particularly decreases prospective political participation among citizens with high levels of conflict avoidance.
Methods and data
Prior to our data collection, all hypotheses were preregistered along with a preanalysis plan at Open Science Framework. Footnote 3 The experiment was conducted in Denmark using a commercial web panel (Voxmeter). The sample size was set to 2,000 participants to maximize statistical power under budgetary constraints. With three condition, the expected group sizes of n=667 yield a statistical power of 95% to find an effect size of just 0.2 standard deviations on the dependent variable (alpha=0.05), conservatively assuming that covariates have zero explanatory power. Overall attrition rate was just 1.0% (among participants exposed to the experimental stimuli, only 0.6% failed to complete the entire survey, and attrition was not significantly associated with the treatment group, p=0.935). The survey questionnaire is included in Appendix A, and a description of the Danish case is found in Appendix B. The final sample is demographically diverse, with 49.0% women, mean age of 52 years (SD=17.0), and the sample also shows substantial variation on political attitudes (see Appendix C for additional sample and population characteristics). Furthermore, while our sample is not a probability sample, we note that treatment effects in survey experiments are generally highly consistent across probability and non-probability samples (Coppock et al., Reference Coppock, Leeper and Mullinix2018).
At the start of the survey, participants were asked standard questions on vote choice, ideological left-right position, and political interest. In addition, conflict avoidance was measured with five agree-disagree items, for example, “I hate arguments” (Bjarnøe et al., Reference Bjarnøe, De Vreese and Albæk2020; Bjarnøe, Reference Bjarnøe2022). Answers to these five items were summarized into an index ranging from zero to 10, with 10 indicating a high level of conflict avoidance. The index has a satisfactory level of internal reliability (Cronbach’s alpha = 0.76).
Experiment
Following the measurements of pretreatment variables, participants were randomly sorted into three groups with equal probability. Participants assigned to group one (control condition) were not shown any social media messages. Participants assigned to group two were shown questioning or mildly critical messages directed at politicians (i.e., messages that were not abusive such as “Wish that you tried to talk to ordinary people in the real world”). Participants assigned to group three were shown abusive messages directed at politicians (designed to be either insulting, sexist, or threatening such as “If you don’t shut your mouth we will come and close it for you”). To heighten the ecological validity of our study, all of the messages used in the treatment material were real messages to politicians on social media (some messages were abbreviated and slightly edited). To ensure that results were not driven by particularities of one specific example of abuse, we used stimulus sampling (Wells and Windschitl, Reference Wells and Windschitl1999), meaning that the messages shown were randomly sampled from a larger pool of messages. All these messages are listed in the questionnaire in Appendix A. Participants in groups two and three were shown three messages sent to politicians and assessed each on a 0–10 scale, ranging from “0: completely unacceptable” to “10: completely acceptable.” This assessment question served two purposes. First, it was included to increase the likelihood of participants reading the treatment text carefully. Second, responses to this question show how participants perceive, respectively, mildly critical and abuse messages. Participants clearly reacted differently to the two types of messages. The mean acceptability rating of the questioning or mildly critical messages was 7.9 on the 0–10 scale (SD=2.5), while the abusive messages had a mean of just 0.7 (SD=1.8).Footnote 4 This demonstrates that the respondents paid attention to the different types of messages and perceived them as intended.
Participants in all groups were subsequently presented with 12 questions on their own prospective political participation (e.g., likelihood of voting, participation in online or offline discussions, and running for office). Most questions were developed specifically for this survey, although they were inspired by previously used questions on retrospective and prospective political participation (Eckstein et al., Reference Eckstein, Noack and Gniewosz2013; Van Deth, Reference Van Deth2014; Elaine and Detenber, Reference Elaine and Detenber2017; Lilleker and Koc-Michalska, Reference Lilleker and Koc-Michalska2017; Ohme et al., Reference Ohme, Vreese and Albæk2018). All these outcome questions were combined into a general index for prospective political participation (Cronbachs alpha=0.86). In addition to this general index for prospective political participation, we also created two sub-indices: prospective political participation on social media (Cronbach’s alpha=0.88) and prospective political participation outside social media (Cronbach’s alpha =0.73). Finally, we analyze prospective political candidacy with the single-item measure of people’s willingness to run as a candidate in municipal elections. Means, standard deviations, and histograms for all dependent variables are included in Tables A1-A2 and Figure A1 in appendix C. While the measures for political participation on social media and, particularly, willingness to run as a candidate are right-skewed, the measures for overall political participation and political participation outside social media (nonsocial media participation) are approximately normally distributed and show a good amount of variation.
Results
In line with the preregistered analyses plan, the tests of the hypotheses were based on regression models (OLS). Starting with hypothesis 1, we test this hypothesis with a model that includes indicator variables for treatment groups and to increase precision of the estimated treatment effects, three covariates expected to be predictive of political participation (gender, age, and political interest).
As Fig. 1 shows, the results of the preregistered models were not in line with hypothesis 1. Compared to the control group, prospective political participation was not lower on any measure among the participants exposed to the abusive messages. Nor did they differ significantly from participants exposed to mildly critical messages.
Figure 1. Main effects on political participation (Preregistered).
Note: Estimates with 95% confidence intervals. Based on models 1–4, Appendix D.
In fact, on all measures, participants exposed to social media messages directed at politicians had higher levels of prospective political participation, although not statistically significantly so. As these results point in the opposite direction of the hypothesized effects, it is worth noting that the covariates in the models generally act as expected. Women and old participants were significantly less likely to be ready to run as candidates (Dahl and Nyrup, Reference Dahl and Nyrup2021). Likewise, political interest was predictive for all measures of political participation.
To additionally test the robustness of these results, we ran additional, exploratory models, respectively, models without any covariates, and models with additional covariates (left-right position and conflict avoidance). These exploratory results are shown in Figure A3, Appendix D. None of these exploratory models show any demobilizing effects of exposure to online abuse. In models without any covariates, exposure to abusive messages actually had a statistically significant positive effect on the overall measures as well as on the measure of non-social media political participation. However, it is important to emphasize that these results are exploratory.
Further, as another exploratory check on the potential negative effects of abusive messages, we repeated the analyses separately for all 12 items measuring prospective political participation (models 13–24 in Appendix D). As shown in Fig. 2, there are no significant demobilizing effects of abusive messages on any of the 12 outcomes. In four cases, exposure to abuse seems to have mobilizing effects: participants exposed to abuse are ostensibly more likely to write to a politician (p=0.025), write a social media post regarding a political issue (p=0.036), write a letter to the editor in a newspaper (p=0.001), and to sign a petition (p= 0.044). It is important to note that these results are based on exploratory analyses, and they are not adjusted for multiple comparisons. In addition, the effect sizes are relatively modest (in the four cases, Cohen’s d was in the range 0.11–0.18). We therefore urge caution when interpreting these apparent effects.
Figure 2. Effects of social media messages across 12 dependent variables (Exploratory).
Note: Estimates with 95% confidence intervals. Based on models 13–24, Appendix D.
We are on firmer ground when concluding on demobilizing effects: in none of the preregistered and exploratory analyses did exposure to online abuse has a statistically significant negative effect. The absence of a statistically significant effect is not proof of no effect, and, in most cases, the confidence intervals of our estimates do not rule out negative effects on the four participation measures. However, exploratory inferiority tests (Lakens et al., Reference Lakens, Scheel and Isager2018) confirm that even small demobilizing effects of just 0.1 standard deviation are highly unlikely (p<0.01) on all four preregistered measures. Thus, we can safely conclude that the abusive comments had no substantively meaningful demobilizing effects at the overall level.
While there are no general demobilizing effects of exposure to online abuse directed at politicians, effects could potentially be different among some subgroups in the populations. To test hypothesis 2 about heterogeneous effects for men and women, we use models with the same variables as the main test of hypothesis 1, as well as the interaction between our treatment indicators and the gender of the participant (models 25–28 in Appendix D). We find that the interactions between the abuse condition and gender are insignificant for all four outcomes: overall participation (p= 0.990), social media participation (p=0.631), participation outside social media (p=0.315) and running for election (p=0.326). These results are thus not supportive of hypothesis 2.
Finally, hypothesis 3 predicted that the effects of online abuse are moderated by participants’ level of conflict avoidance. This is tested in models with the same variables as the test of hypothesis 1, as well as interaction terms for the treatment indicators and the continuous measure of conflict avoidance (models 29–32 in Appendix D). It is worth noting that conflict avoidance in itself has a clear and statistically significant correlation with political participation: while participants with very low levels of conflict avoidance (here defined as the 5th percentile) have a mean score of 4.2 (95% CI: 4.1–4.3) on the overall index of political participation, participants with a very high level of conflict avoidance (95th percentile) have a mean score of just 3.2 (95% CI: 3.0–3.3).Footnote 5 However, the interactions between the abuse condition and conflict avoidance are statistically insignificant for all four preregistered measures of political participation: overall participation (p= 0.147), social media participation (p= 0.374), participation outside of social media (p= 0.180), and running for election (p= 0.177). In sum, there is no support for hypothesis 3.
Conclusion and discussion
Online abuse of politicians does not seem to turn ordinary citizens off politics. Our results suggest that citizens exposed to real-life examples of politicians being abused online are no less likely – or perhaps even more likely – to participate in politics. While we underscore that the mobilizing effects of observing online abuse were only found in exploratory analyses, there may indeed be theoretical reasons to suggest such effects. Exposure to incivility has been found to spur general arousal or anger, which can potentially mobilize politically (Mutz, Reference Mutz2007; Valentino et al., Reference Valentino, Brader, Groenendyk, Gregorowicz and Hutchings2011; Sydnor, Reference Sydnor2019). Even though online abuse tends to be much harsher than mere incivility, the same mechanism may be at play here.
It is important to note that our experiment only tested the effects of abuse directed at politicians. Thus, our experiment does not rule out that people may be politically demobilized if the abuse of politicians is accompanied by abuse of other groups, e.g., groups that citizens themselves belong to. Future research could thus investigate how abuse of politicians is interlinked with abuse directed at other targets. Another potential criticism of our experimental design could be that it fails to capture the demobilizing effects of online abuse because respondents had been exposed to online abuse of politicians multiple times before participating in our experiment, thus rendering the experiment’s manipulation effectless. While we cannot rule out such pretreatment (Druckman and Leeper, Reference Druckman and Leeper2012), our findings nevertheless show that (additional) exposure to online abuse of politicians does not scare ordinary citizens away from politics.
This is a normatively important conclusion, as it means that there is little reason to shy away from publicly raising the problem of online abuse of politicians. There is seemingly limited risk that such debates can adversely affect citizens’ political participation, and there are still strong reasons to consider online abuse to be a serious problem. First, our results do not invalidate the studies suggesting that online abuse is a serious threat to politicians’ well-being and motivation to stay in politics (James et al., Reference James, Sukhwal, Farnham, Evans, Barrie, Taylor and Wilson2016; Pedersen et al., Reference Pedersen, Petersen and Thau2024a; Collignon and Rüdig, Reference Collignon and Rüdig2020). Second, while one can reasonably argue that there should be room for some incivility in politics (Mutz, Reference Mutz2015; Chen et al., Reference Chen, Muddiman, Wilner, Pariser and Jomini Stroud2019; Sydnor Reference Sydnor2019), it is very difficult to argue that outright abuse such as sexist comments and death provides anything of value to the democratic discourse. Thus, online abuse of politicians is still a serious problem for democracies, and a problem which we should not be afraid to talk about.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/XPS.2024.20
Data availability
Replication materials are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, doi: 10.7910/DVN/SXOYQR (Pedersen et al., Reference Pedersen, Petersen and Thau2024b).
Acknowledgments
This research was funded by Trygfonden (Grant number 153435). We are grateful to the Editors and the anonymous reviewers for their constructive feedback. This study was previously presented at the 2023 annual conference of the Danish Political Science Association and at seminars at The Danish Center for Social Science Research as well as Institute for Social Research in Norway. The authors are grateful for the helpful and constructive comments from participants on these occasions.
Competing interests
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This work is part of the research project ‘Online Abuse of Danish Politicians’ funded by Trygfonden (grant ID: 153435).
Ethics statement
This study was approved by the Research Ethics Committee at The Danish Center for Social Science Research (2023/9). The authors affirm that the research adheres to APSA’s Principles and Guidance for Human Subjects Research. Informed consent was obtained from all participants. The survey company Voxmeter compensates participants with points, which can be used for lotteries, goods, and charitable donations.
Open access
