Introduction
An emerging body of literature argues that the increasingly frequent strong electoral performances by radical right parties (Mudde Reference Mudde2016), as well as the rise of extreme behavior such as hate crimes (FBI 2023), are partially triggered by changing social norms (Valentim Reference Valentim2024b). According to this argument, the normalization of radical right parties has paved the way for increased expression of counter-normative preferences, behaviors, and endorsement of extremist political parties (Tankard and Paluck Reference Tankard and Paluck2016; Bursztyn, Egorov, and Fiorin Reference Bursztyn, Egorov and Fiorin2020; Romarri Reference Romarri2020; Valentim Reference Valentim2021; Albornoz, Bradley, and Sonderegger Reference Albornoz, Bradley and Sonderegger2022; Gul Reference Gul2023). Strengthening anti-radical right norms, then, could be an effective way of reducing radical right behavior.
Norms depend on one’s perceptions of what is deemed acceptable in their social environment (Bicchieri Reference Bicchieri2016). Since these second-order expectations are rarely observed, citizens rely on cues that help them generate norm perceptions. Identifying and manipulating those cues can be a way of strengthening norms against radical right behavior. Existing literature has highlighted three sources of cues that might affect norm perceptions. First, cues can come from the behavior of other citizens that one interacts with in their daily life (Blinder, Ford, and Ivarsflaten Reference Blinder, Ford and Ivarsflaten2013; Harteveld et al. Reference Harteveld, Dahlberg, Kokkonen and van der Brug2019; Ammassari Reference Ammassari2023; Oshri et al. Reference Oshri, Harsgor, Itzkovitch-Malka and Tuttnauer2023; Alvarez-Benjumea and Valentim Reference Alvarez-Benjumea and Valentim2024). Second, cues can emanate from media discourse (van Heerden and van der Brug Reference Van Heerden and van der Brug2017; van Spanje and Weber Reference Van Spanje and Weber2019; Bolet and Foos Reference Bolet and Foos2023). Finally, cues may be signaled by the behavior of political elites (Clayton et al. Reference Clayton, Davis, Nyhan, Porter, Ryan and Wood2021; de Jonge and Gaufman Reference De Jonge and Gaufman2022; Krause, Cohen, and Abou-Chadi Reference Krause, Cohen and Abou-Chadi2023; Axelsen Reference Axelsen2024). Experimentally, researchers have used interventions from these actors to increase (or decrease) the salience of social stigma surrounding radical right parties (e.g., Harteveld et al. Reference Harteveld, Dahlberg, Kokkonen and van der Brug2019; Bursztyn, Egorov, and Fiorin Reference Bursztyn, Egorov and Fiorin2020), though there is little systematic validation of these primes.
Following this literature, we build and validate experimental treatments designed to manipulate stigma and assess their effectiveness in strengthening anti-radical right norms. We use a well-powered pre-registered survey experiment that randomly assigns 5,526 Spanish adults to one of four groups. In three groups, individuals are treated with cues from sources identified in the literature (peers, media, and political elites) meant to reinforce stigma against the radical right party Vox. The fourth arm is a control group that receives a neutral prime about the party. Afterwards, participants answer a series of questions assessing the strength of the political stigma surrounding the party, including items that capture first- and second-order perceptions of the acceptability of Vox and the likelihood of sanctioning public displays of support for Vox (Bicchieri Reference Bicchieri2016).
Contrary to our expectations, these primes failed to induce stigma against Vox. This suggests that the ability of relatively light-touch vignette interventions to increase stigma against the radical right is fairly weak. In some cases, we even find that attempts at stigmatization seem to backfire, leading to greater reported acceptability of the radical right. These findings offer two key implications for experimental researchers and scholars of norms and parties. First, we show that researchers should not rely on this type of cue-based vignettes to raise the social stigma surrounding a party. Accordingly, we call for careful validation when implementing treatments aimed at re-stigmatization. Second, our results suggest that once a party has largely been normalized, it is difficult to “re-stigmatize” it. Thus, party normalization may be best conceived of as a “one-way street,” where normalization cannot be easily countered and re-stigmatization is unlikely.
Hypotheses
Radical right parties are often stigmatized at their inception, due to historical legacies, the extremity of their positions, and anti-liberal elements of their ideology. Stigma, in this context, is understood as a social norm (Valentim Reference Valentim2024a), encompassing both first- and second-order judgments about the acceptability of radical right parties and the likelihood of facing sanctions for public endorsement (Bicchieri Reference Bicchieri2016).Footnote 1 However, this normative pressure typically erodes over time, especially as far-right parties gain traction within the political system and receive tacit or overt acceptance from other parties and the public.
The literature identifies three leading actors that may cue the acceptability of these parties. First, citizens may update their views based on perceptions of those in their social networks, as individuals typically attempt to conform to the attitudes and behaviors of their peers (Blinder, Ford, and Ivarsflaten Reference Blinder, Ford and Ivarsflaten2013; Harteveld et al. Reference Harteveld, Dahlberg, Kokkonen and van der Brug2019; Ammassari Reference Ammassari2022; Oshri et al. Reference Oshri, Harsgor, Itzkovitch-Malka and Tuttnauer2023; Dinas, Fouka, and Schläpfer Reference Dinas, Fouka and Schläpfer2021; Alvarez-Benjumea and Valentim Reference Alvarez-Benjumea and Valentim2024). Second, such updates may stem from observing media cues, which play an important role in setting agendas and defining the bounds of “normal” politics (van Heerden and van der Brug Reference Van Heerden and van der Brug2017; van Spanje and Weber Reference Van Spanje and Weber2019; Bolet and Foos Reference Bolet and Foos2023). Finally, individuals may update their views based on cues from politicians’ statements (e.g., issue positions) and political party actions (e.g., coalition intentions), as these actors also hold significant agenda-setting power (Clayton et al. Reference Clayton, Davis, Nyhan, Porter, Ryan and Wood2021; de Jonge and Gaufman Reference De Jonge and Gaufman2022; Krause, Cohen, and Abou-Chadi Reference Krause, Cohen and Abou-Chadi2023; Axelsen Reference Axelsen2024). But to what extent can these cues increase stigma against a radical right party? If cues of stigma operate as intended, we should observe that first- and second-order expectations and normative evaluations can be moved to reflect a greater generalized stigmatization of the radical right, leading to our first hypothesis:
-
• H1A: Priming a source-specific stigma will increase generalized stigma of Vox.
H1A assumes a homogeneous effect of our treatments on generalized stigma. One might expect that such effect is conditional on ideology (Tankard and Paluck Reference Tankard and Paluck2016). Indeed, past evidence documents that leftists often already deem radical right parties unacceptable (Golder Reference Golder2016; Dinas, Martínez, and Valentim Reference Dinas, Martínez and Valentim2024). This might make it harder to shift perceptions of left-wing individuals due to ceiling effects. We explore this possibility by testing the following hypothesis:
-
• H1B: Among respondents ideologically placed in the center or right, priming a source-specific stigma will increase generalized stigma of Vox.
It should be noted, however, that our main quantity of interest throughout is an average treatment effect, rather than a treatment effect conditional on ideology. This is because the entire population, which includes individuals across the political spectrum, plays a role in constituting and upholding norms. Therefore, strengthening perceived stigma against the radical right among individuals across the political spectrum – left-wing, centrist, and right-wing – can influence the anticipated prominence of radical right behavior in the future.Footnote 2 Moreover, using a sampling strategy that does not target specific ideological types allows us to examine to what extent these different movements – if they exist at all – cancel each other out.
While the main task of this project is to validate treatments stigmatizing the radical right, we are also interested in comparing the efficacy of different source cues. Recent observational evidence documents mixed results concerning the effects of elite-led acceptance or stigmatization strategies, such as coalitions or cordon sanitaires (van Spanje and Weber Reference Van Spanje and Weber2019; Favero and Zulianello 2023). At the same time, peer-based stigmatization may be more efficient in deterring radical right support, as peers are the more proximate and relevant reference group in most individuals’ everyday lives (Ammassari Reference Ammassari2024). Accordingly, cuing stigma via a vignette related to one’s peers may have stronger effects than media-led or political elite-led stigmatization,Footnote 3 leading to the hypothesis:
-
• H2: Stigma cued via peers is more effective in increasing generalized stigma than stigma cued via media or political elite sources.
Experimental design
To validate and compare the efficacy of various cues of stigma, we conducted a four-arm survey experiment in Spain, administered online by the survey firm 40dB. In total, we collected data from 5,526 adult respondents between June 17 and July 3, 2024. Our sample roughly matches the Spanish population on gender, age, and autonomous community (region) (see Section A1 for further details).
Our treatments cued political stigma surrounding Vox, the first radical right party in Spain to obtain parliamentary representation after the transition to democracy in the late 1970s. Despite currently being the third-largest party in the Spanish Congress (with 33 MPs), Vox is widely perceived as a party that challenges the fundamental principles of liberal democracy and is deemed unacceptable by a significant segment of the electorate.Footnote 4 As argued by Alvarez-Benjumea and Valentim (Reference Alvarez-Benjumea and Valentim2024), Vox’s status of partial normalization and stigmatization offers latitude to plausibly manipulate, using a survey experiment, the perceived normative expectations associated with this party.
Figure 1 depicts the structure of our experimental design. First, we gathered data on the socio-demographic and ideological profiles of respondents. We then randomized respondents into one of four treatment arms. Respondents were mostly balanced on pre-treatment observable covariates across treatment groups (see Section A2).Footnote 5 Also, we find no evidence of differential attrition by treatment assignment (see Section A3). In the control condition, respondents received a neutral message that primed them to think of Vox, void of information related to social norms surrounding the party. We included three treatment groups, each representing a specific source cuing social stigma against Vox. The treatment arms were designed to be as similar as possible to ensure information equivalence and vary only in the source of the stigma. Table 1 displays the text included in each treatment arm (translated into English).
Figure 1. Consort diagram.
Note: Diagram depicts the general flow of the survey experiment, which follows a between-subjects design. Respondents were randomized into one of four groups with equal probability before responding to the same set of post-treatment outcomes.
Table 1. Vignettes corresponding to each treatment arm
After respondents were exposed to the treatment, they answered a set of questions tapping first- and second-order perceptions of political stigma surrounding Vox. These include (1) personal perception of the acceptability of Vox, (2) perception of the acceptability of Vox among cues [peers/media/political elites], (3) personal willingness to sanction Vox supporters, and (4) expectations that others would sanction Vox supporters.Footnote 6 Outcomes were all measured on 0–10 scales and were re-scaled such that higher values indicate greater levels of stigma.
One possible concern is that responses to our survey items were affected by preference falsification (Kuran Reference Kuran1995; Valentim Reference Valentim2024a). In making respondents more aware of norms, our treatments could make them insincerely respond to outcome items. However, preference falsification in survey item responses can be seen, itself, as a measure of the social norms in place and their strength (Valentim Reference Valentim2021). As such, even if the responses provided were somewhat affected by this phenomenon, this would imply that the treatments are affecting norms in the expected way.
Testing hypotheses
To test our hypotheses, we estimate the average treatment effect of cuing source-specific stigma on participants’ perceived (generalized) stigma by using a simple difference-in-means estimator.Footnote 7 H1A and H1B consider differences between a source-specific stigma prime and the control group on an indexed outcome of generalized stigma, operationalized as the average stigma across the four outcomes. H1B, which posits conditional effects by pre-treatment ideology, is tested by splitting the sample by those identifying as left-wing (between a 0:4 on the self-reported ideological scale, measured pre-treatment) or right-wing (between a 5:10).Footnote 8
To examine which treatments increase stigma the most (H2), we compare the difference in generalized stigma among the peer-cued treatment group and a combined media and political elite-cued treatment group. Our intuition is that peers are a more relevant reference group than the media or political elites, meaning that the peer-based prime is more effective than the remaining treatments.Footnote 9 In total, we powered off detecting significant effects from 4 main hypotheses (3 tests for H1A, 1 test for H2). Our pre-analysis plan (Section A10) details our statistical power, which we argue is quite strong. We reduce the threat of type II errors by using Benjamini-Hochberg corrections for the 4 pre-registered hypotheses.
Results
We proceed with our main results in two stages. First, we examine the effects of the stigmatization primes against the control condition (H1A and H1B). Figure 2A shows the effects of each treatment on specific second-order perceptions of a group’s acceptability of Vox, which serves as a substantive manipulation check.Footnote 10 Figure 2B shows the effects of each prime on generalized stigma, included when separating the sample by “right” and “left.”
Figure 2. Main results: manipulation and validation.
Note: All estimates are depicted with 90 (thick) and 95% (thin) confidence intervals.
As a substantive manipulation check, we examine the differences-in-means between a source-specific stigma prime and the control condition on the corresponding second-order expectation. For example, the estimate in the first row of Figure 2A shows the effect of the peer-cued stigma on perceptions that Spanish citizens deem Vox as unacceptable. If our manipulations were interpreted as convincing, we would expect the treatment to increase the perception of the corresponding second-order perception. We find that this is true for “media” cued stigma, but less so for “political elite” cued stigma, whose effect is in the expected direction but does not reach conventional levels of statistical significance (p = 0.137).Footnote 11 However, we find little evidence that the “peer” treatment manipulated second-order perceptions of peer normative evaluations. These results suggest that, on average, respondents did not update their expectation that other citizens (nor, to a certain extent, political elites) deem Vox unacceptable after being primed with information making this case.
One possible explanation for these discrepancies is that individuals may not perceive these primes to be equally realistic. It may be that citizens have better information about how people in their networks (and the political elites they are exposed to) view the radical right, making it more difficult to shift perceptions and making them relatively stable.Footnote 12
Next, Figure 2B presents the effect of each prime on generalized stigma, which indexes first and second-order perceptions of Vox’s acceptability as well as willingness to sanction public displays of support for Vox. Given the results from Figure 2A, we should note that this estimate does not necessarily reflect the effect of source-cued stigma. In partially failing to validate this first stage (e.g., shifting the perception of one source-cue as stigmatizing the radical right), our main results are better understood as capturing the effect of exposure to an external message about group-cued stigma.
Across each actor, we find no evidence supporting H1A: priming a source-specific cue of stigma does not increase generalized stigma against Vox. If anything, our evidence is consistent with a backlash effect, namely exposing participants to vignettes meant to cue stigma can actually reduce stigma against the radical right. When primed with peer-cued stigma, we notice a negative and statistically significant treatment effect (first row of Figure 2B). This backlash effect appears to be driven by left-wing respondents, though it should be noted that these respondents have considerably more “room to move,” as left-wing respondents in the control condition display higher levels of generalized stigma than right-wing respondents in this condition.Footnote 13 Similarly, we find that left-wing respondents in the “media” treatment group exhibited a backlash, where they reported lower levels of stigma towards Vox compared to the left-wing respondents in control group.
Together, we find no evidence for H1B, which proposes that effects are most likely among right-wing respondents.Footnote 14 Earlier, we theorized that individuals on the left might be less responsive to the vignettes, given their pre-existing tendency to perceive the radical right as stigmatized. Contrary to this expectation, our findings suggest the opposite trend.Footnote 15
Disaggregating the generalized stigma index shows a more nuanced picture.Footnote 16 Figure 3 plots the estimated effect of each treatment arm when compared to the control condition on first-order normative evaluation of Vox, the average second-order normative evaluations concerning all three cue-sources, first-order willingness to sanction public support for Vox, and second-order expectations of others’ willingness to sanction public support for Vox.
Figure 3. Replicating Figure 2B with disaggregated index.
Note: All estimates are depicted with 90 (thick) and 95% (thin) confidence intervals.
We observe a backlash to the “peer” and “media” cued stigma treatments for first-order normative evaluations, where individuals are less likely to rate Vox as unacceptable compared to the control group. Interestingly, second-order normative expectations move following a heterogeneous pattern. Those on the left appear the most likely to reject the treatment and identify less second-order normative stigma against Vox after being primed. On the right, the treatments generally induce greater second-order stigma. This difference could be a product of prior perceptions of unfair treatment: those on the right may feel that Vox is overly or unjustly stigmatized, allowing them to be amenable to the claim that peers and the media treat Vox unfairly. Those on the left, on the other hand, may feel that many peers and the media environment have been too normalizing or welcoming to Vox, therefore rejecting a prime suggesting that the party faces significant stigma. These effects do not translate into our sanctioning outcomes, though, which appear more stable and less sensitive to the treatments.
Altogether, these results suggest that re-stigmatizing the radical right is not a straightforward endeavor. For the most part, our data suggests that (1) light-touch vignette interventions aimed at stigmatizing the radical right are not particularly believable and that (2) highlighting external stigma against the radical right does quite poorly for leading individuals to adopt congruent ideas about the acceptability of such parties. In some cases, they can even lead to a backlash.
As we fail to validate these treatments our question about the comparative efficacy of treatments can be seen in a new light. Rather than identifying treatments that are most effective at stigmatizing the far right (as referenced in H2), we can instead ask: which cues of stigma are most likely to backfire?
To answer this question, Figure 4 examines the difference in means for the generalized stigma index between different treatment group comparisons (as denoted on the y-axis). Opposite to H2, we find that peer-cued stigma is more likely than other cues of stigma to backfire. In other words, when individuals read about peer-cued stigma against the radical right (as opposed to cues from other actors – mainly, political elites), they are more likely to report that the radical right is acceptable.
Figure 4. Comparative efficacy of treatments on generalized stigma.
Note: All estimates are depicted with 90 (thick) and 95% (thin) confidence intervals.
Conclusion
One of the most pressing political questions of recent times is how to tackle growing radical right behavior across Western democracies. In our experiment, we find that treatments priming anti-radical right norms are probably not the best solution, since they overwhelmingly fail to stigmatize radical right parties.
This conclusion has several implications for existing literature. First, it highlights that, while norm change may have helped the rise of radical right parties, norm erosion cannot easily be reverse-engineered. The process of normalization of the radical right hinged on undoing the preference falsification in which radical right voters were previously engaging (Valentim Reference Valentim2024b). After radical right parties have become successful and these voters learn that many in their society share their views, it is hard to bring them back to a state of “not knowing.”
Our results also highlight that, in some cases, efforts to stigmatize the radical right may even backfire. One interpretation of this finding is that, in a setting like the one we study – where the radical right is already partially normalized – being exposed to information about social norms that is at odds with one’s everyday perceptions can make respondents feel that the party is being unfairly treated. As a response, they can come to perceive such party as being more (not less) socially acceptable (e.g., Brehm et al. Reference Brehm, Stires, Sensenig and Shaban1966).
Finally, an important caveat is that, unlike other norms-based treatments (e.g., Bursztyn, Egorov, and Fiorin Reference Bursztyn, Egorov and Fiorin2020), our vignettes do not expose respondents to any actual information about others’ beliefs. This could be another reason driving our null and backlash findings. If individuals did misperceive the views of others, treating them with accurate information about the distribution of preferences in society might still have an effect (Bicchieri Reference Bicchieri2016). This project underscores the need for further validation of stigmatization treatments and serves as a cautionary note for experimentalists attempting to manipulate stigma. This point, more broadly, emphasizes the difficulty of reverse-engineering processes of normalization when such stigma has been reduced.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2025.10007
Data availability
The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/QPTLBA.
Acknowledgements
We thank four anonymous reviewers and the JEPS associate editor for comments on previous versions of this project. This study is part of the Georgetown University project “Inequality and Governance in Unstable Democracies–The mediating Role of Trust,” implemented by a consortium led by the Institute of Development Studies (IDS). The support of the UK Economic and Social Research Council (ESRC grant ES/S009965/1) is gratefully acknowledged. We acknowledge the use of Grammarly and OpenAI’s GPT-4 for assistance with grammar and language editing. The content, analysis, and interpretations presented in the article are entirely our own. Any remaining errors are ours.
Ethics statement
This project was approved by the ethics boards of Georgetown University (ID: STUDY00007955) and the University of Wisconsin-Madison (ID: 2024-0656). The other authors involved in the project were not included in any data collection activities and did not have access to the original data until it was cleaned and fully anonymized. This research adheres to APSA’s Principles and Guidance for Human Subjects Research and did not include the use of deception or any substantial potential harm. Participants gave voluntary and willing consent before taking the survey and were compensated via an existing agreement with the survey firm.
Open access
