Over the past decade, international relations (IR) research has seen a boom in published survey experiments, but due to limitations in data availability researchers often rely on samples drawn from the mass public. This practice risks turning IR questions into public opinion questions without exploring preferences of other important actors, such as foreign policy experts. While the members of the public form their own opinions about foreign policy on occasion (Kertzer and Zeitzoff, Reference Kertzer and Zeitzoff2017), they often defer to experts (Jost and Kertzer, Reference Jost and Kertzer2024), who tend to have more freedom to pursue their preferred policies than professionals in other fields do. This raises an important question: to what extent can researchers generalize survey-experimental findings from the mass public onto the attitudes and behaviors of foreign policy experts?
A longstanding IR question that has been prominently investigated with survey experiments employing the samples of the mass public is democratic peace theory (Tomz and Weeks, Reference Tomz and Weeks2013). While the standard version of the theory posits that public opinion is a sufficient explanation for democratic peace, some contributions have argued that it rests on the interplay between the public and elites (Owen, Reference Owen1994; Goldsmith et al., Reference Goldsmith, Semenovich, Sowmya and Grgic2017; Tomz et al., Reference Tomz, Weeks and Yarhi-Milo2020). This latter interpretation is buttressed by recent findings that public preferences about the use of force in autocracies are geared away from conflict with democracies as well (Bell and Quek, Reference Bell and Quek2018), yet it does not seem to affect the behavior of authoritarian elites in the same way. Indeed, not being bound by preferences of the public, authoritarian countries attack democracies no less than fellow autocracies (Maoz and Russett, Reference Maoz and Russett1993; Gartzke, Reference Gartzke1998).
Our goal in this research note is to assess the degree to which the opinions of foreign policy experts diverge from those of ordinary citizens, and we engage with this question by simultaneously investigating preferences of these two groups on democratic peace. Specifically, we implement two identical conjoint survey experiments: one on a standard sample of U.S. adults and another on a sample of U.S. Department of State employees. In these experiments, mass and expert respondents are presented with profiles of hypothetical countries described in terms of six randomized attributes, including political regime, and asked about their approval of military strikes against those countries.
Results of the experiments demonstrate that preferences against military conflicts with other democracies among foreign policy experts are relatively stronger than among members of the public. More broadly, we find foreign policy experts in the United States to have preferences regarding matters of war and peace that better align with liberal theories of world politics than those of the public. Our findings call for a renewed attention to the generalizability of IR survey experiments from public to expert preferences.
1. Public, experts, and foreign policy
We test the validity of generalizing foreign policy preferences from mass attitudes by examining one of the most studied phenomena in IR: democratic peace. Multiple studies have explored public preference against conflicts with other democracies using survey experiments, so its attitudinal micro-foundations are well established (Lacina and Lee, Reference Lacina and Lee2013; Tomz and Weeks, Reference Tomz and Weeks2013; Bell and Quek, Reference Bell and Quek2018; Bakker, Reference Bakker2020). Yet the question of how and to what extent public opinion matches expert opinion on this topic is not nearly so well studied.
There are several reasons why identifying the possible gap between mass and expert preferences is analytically relevant. Unelected experts play a prominent role in policy making due to their crucial role in providing information and briefings on foreign affairs to elected officials (Jacobs and Page, Reference Jacobs and Page2005; Jost et al., Reference Jost, Kertzer, Min and Schub2024). While politicians may be affected by public opinion because they care about reelection (Tomz et al., Reference Tomz, Weeks and Yarhi-Milo2020), unelected bureaucrats do not face the same constraint—and sometimes can resist pressure from elected officials (Drezner, Reference Drezner2019). Ultimately, experts, especially co-partisans, may be able to shape public opinion on international conflict, and this ability is particularly pronounced at its initiating phase (Baum and Groeling, Reference Baum and Groeling2010; Gelpi, Reference Gelpi2017).
By focusing on preferences of the bureaucracy, we also provide a synthesis of the democratic peace literature with studies that focus on the so-called “foreign policy Blob” (Porter, Reference Porter2018; Walt, Reference Walt2018), which is supposedly biased in favor of liberal internationalism (Chaudoin et al., Reference Chaudoin, Milner and Tingley2010; but cf Kupchan and Trubowitz, Reference Kupchan and Trubowitz2007, Reference Kupchan and Trubowitz2010). Putting the normative debate aside, conclusions regarding the importance of the Blob should be based on appropriately specified counterfactuals (Jervis, Reference Jervis2020). To this end, we estimate the degree to which the preferences of unelected experts in the field of foreign policy, which are the primary focus of this literature, exhibit liberal bias vis-a-vis the public.
2. Disentangling public and experts’ preferences
We consider three possible options of how mass and expert preferences on democratic peace can relate to each other. First, experts can have weaker preferences than the public, as their primary focus is on following political directives without following their personal opinions. Second, experts can have stronger preferences than the public, as they are part of the, generally more liberal and higher educated, foreign policy Blob (Chaudoin et al., Reference Chaudoin, Milner and Tingley2010). Third, it is possible that preferences of the public and experts are broadly the same—after all, experts are recruited from the public at some point in their careers. Since the existing literature does not offer explicit support for any one of these possibilities, we are agnostic about which of the three options is more likely.
We explore these options in two identical conjoint experiments that are implemented in parallel in the mass sample and the sample of foreign policy experts. The external validity of preference estimates obtained from conjoint experiments has been demonstrated using real-world behaviors on both aggregate and individual level as the benchmarks (Hainmueller et al., Reference Hainmueller, Hangartner and Yamamoto2015; Jenke et al., Reference Jenke, Bansak, Hainmueller and Hangartner2021). From the survey-methodological perspective, conjoint experiments are resilient to data quality issues such as satisficing (Bansak et al., Reference Bansak, Hainmueller, Hopkins and Yamamoto2018). In addition, expert samples may be particularly prone to social desirability bias (Naurin and Öhberg, Reference Naurin and Öhberg2021), and conjoint experiments help to mitigate it (Horiuchi et al., Reference Horiuchi, Markovich and Yamamoto2022). Finally, the possibility to randomize the order of the attributes of interest—and thus directly compare their effects’ magnitudes—is an important benefit of the conjoint design vis-à-vis the vignette design (Bansak et al., Reference Bansak, Hainmueller, Hopkins, Yamamoto, Druckman and Green2021).
In our experiments, respondents are presented with profiles of hypothetical countries and asked about their approval of military strikes against them. The conjoint-experimental design allows us to independently randomize both political regimes of the target countries and other attributes that the government type may signal. The fact that respondents make multiple choices also helps with statistical power in expert samples that tend to be relatively small. Because both types of respondents complete the same tasks, effect sizes, and directions are directly comparable, allowing us to determine to which extent mass and expert preferences are the same.
3. Data and methods
Between February and June of 2023, we fielded original online survey-experimental studies on both a sample of the mass public and a sample of foreign policy experts. Respondents for the mass sample were recruited using Lucid Theorem, a platform with a participant pool that is demographically close to the national probability benchmarks (Coppock and McClellan, Reference Coppock and McClellan2019). To reach the expert sample, we used the professional networking social media platform LinkedIn (for a similar data collection strategy, see Clark, Reference Clark2021). We used the website’s advertisement tool to send invitations and survey links to individuals who had college education or higher and were employed by the U.S. Department of State. Our recruitment of expert respondents followed the recommended best practices (Kertzer and Renshon, Reference Kertzer and Renshon2022). The final samples included 985 mass respondents and 147 expert respondents.
The main part of the survey was a conjoint experiment, in which respondents were presented with paired profiles of hypothetical countries and asked military action against which country was more justified (i.e., we used a forced choice conjoint design). Each respondent made six choices and thus saw a total of 12 country profiles. The profiles were described using the following attributes: reason for military action, political regime, dominant religion, status as a U.S. ally, military capability, and amount of trade with the United States. Reason for military action was always presented first whereas the order of other attributes was randomized between respondents (but kept the same for each individual respondent). Attribute values were fully and independently randomized with uniform distributions (i.e., all potential attributes had equal probabilities of being presented). See Table 1 for all attributes and potential values and Figure 1 for a sample pair of country profiles.
Figure 1. Example of country profiles as presented to respondents.
Table 1. Attributes for country profiles in the conjoint experiment
“Reason for military action” attribute always presented first. Order of all other attributes randomized between subjects. All attribute values randomized independently with uniform distributions.
After the conjoint task, respondents completed the shortened ethnocentrism battery (Bizumic and Duckitt, Reference Bizumic and Duckitt2012), and the militant vs. cooperative foreign policy attitudes battery (Endicott, Reference Endicott2020). For the mass sample, education and partisanship data was provided by the panel. For the expert sample, questions on partisan identification were asked in the survey.Footnote 1 See Section A of Supplementary Material for the questions.
4. Results
We start from estimating the AMCEs using the standard method: OLS regression with errors clustered by respondent (Hainmueller et al., Reference Hainmueller, Hopkins and Yamamoto2014). Results for the mass sample and the expert sample are presented in Figure 2. They show that a country’s status as a U.S. ally and democratic political regime make respondents in both samples significantly less likely to justify military action against it. Among expert respondents, military action against countries with higher amounts of trade with the United States is also less likely to be justified. Interestingly, mass respondents are more likely to justify a military strike against a country with stronger military capacity. This could be due to the Thucydides Trap, but we leave this puzzle for future studies to unpack. The effect of a country’s dominant religion is not significant in either sample. The effects of stated reasons for potential military action are not directly comparable to other attributes due to being always presented first and having more potential values. However, it is worth noting that in both samples sponsoring terrorism is seen as a more justified reason for a military strike than either human rights abuse or a nuclear program. Foreign policy experts also see human rights abuse as the least justifiable reason for a military strike.Footnote 2
Figure 2. Results of the conjoint experiment: estimated effects on justification of military action.
Even though the directions of preferences are mostly similar, the magnitudes of almost all effects are substantially higher in the expert sample compared with the mass sample. Table 2 provides exact estimates of differences in AMCEs—and thus in corresponding preferences—between the mass sample and the expert sample with the corresponding significance tests. The differences between the two samples are also jointly significant on the 99.9% confidence level (F 5, 1131 = 15.92, P < .001).
Table 2. Differences in effects on justification of military action between the mass sample and the expert sample
Standard errors in parentheses
* P < .05, **P < .01, ***P < .001.
As the second step of the analysis, we explore heterogeneity of preferences with respect to the core attribute of interest, political regime, in the mass sample.Footnote 3 Table 3 presents the corresponding AMCEs across respondents’ ethnocentrism and education.Footnote 4 Results show that preferences for military strikes against democracies (as opposed to autocracies) significantly interact with ethnocentrism and education.Footnote 5 Specifically, those who are low in ethnocentrism and respondents with college education are less likely to justify military strikes against democracies. These findings can at least partially explain the greater magnitudes of effects in the expert sample, who are lower on ethnocentrism than the mass sample and have higher education.Footnote 6
Table 3. Effects of political regime (democracy vs. autocracy) on justification of military action by respondents’ ethnocentrism and education in the mass sample
Standard errors in parentheses
* P < .05, **P < .01, ***P < .001.
5. Conclusion
The goal of our study has been to compare the preferences of the mass public and foreign policy experts regarding conflicts with other democracies—and thus the generalizability of surveys of the public and foreign policy experts. We find that both mass and expert respondents are less likely to justify military actions against democracies than with autocracies, but this preference is much more pronounced among foreign policy experts. This result echoes previous arguments regarding the importance of liberal values among foreign policy decision makers for democratic peace (Owen, Reference Owen1994), as well as findings about the direction and magnitude of the elite-public gap. Specifically, our interactive results of the mass public sample compared to the expert sample also buttress an earlier finding that a large part of the elite-public gap seems to be down to different levels of education and social attitudes (Kertzer, Reference Kertzer2022).
When interpreting these results, it is necessary to remember that preferences about military actions are generally stronger in the expert sample, and that expert preferences align more strongly not just with democratic peace theory, but other liberal theories of war as well. For instance, the preference against military actions against U.S. allies is also more pronounced in the expert sample than in the mass sample, as is the effect of international trade volume. This is probably not surprising given that the matters of IR are much closer to the daily lives of the Department of State employees than they are to those of ordinary citizens. We are agnostic as to whether these results imply that IR concepts like democratic peace or free trade’s pacifying effects are correct, or that we have merely touched upon a case of groupthink among the foreign policy bureaucracy as argued by those who criticize the “Washington Blob” (Porter, Reference Porter2018; Walt, Reference Walt2018).
A potential limitation of our experimental design is its highly stylized character: the conjoint task does not perfectly reflect real-world decision-making situations in foreign policy. However, recent findings demonstrate that the hypothetical character of tasks used in survey experiments does not impact their substantive results (Brutger et al., Reference Brutger, Kertzer, Renshon and Weiss2022). In addition, the chosen design does not threaten the finding regarding stronger preferences in the expert sample. As demonstrated in expert respondents’ feedback presented in Section D of Supplementary Material, members of that group should be more sensitive to the stylized character of our experimental design. Even if skepticism about the design suppressed AMCEs in the expert sample, that would lead us to underestimate the differences in preferences between the two samples. It means that, if anything, our estimate of the gap between mass and expert respondents on foreign policy preferences is conservative.
In the light of our findings, future studies should further explore the origins and implications of experts’ preferences. For instance, an investigation of the selection process for the foreign policy bureaucracy in democratic countries—and whether adherence to liberal values is rewarded—may be particularly interesting. Scholars may also engage in the comparative study of preferences among foreign policy experts in democratic and authoritarian regimes. If these preferences diverge and foreign policy experts in authoritarian countries are not opposed to conflict with democracies, it may have very important implications for learning about the link between the mass public and elite decision makers in these countries. Overall, the results of our study call for renewed investigation of the mechanisms behind democratic peace and the role that permanent bureaucracies may play in it as well as in foreign policy decision making.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2025.10026. To obtain replication material for this article, please visit https://doi.org/10.7910/DVN/F0TFGQ.
Acknowledgements
Data collection was supported by a Quantitative Collaborative Faculty Seed Grant at the University of Virginia. An earlier version of this project was presented at the Annual Meeting of the Midwest Political Science Association. We are grateful to Patrick Larue and Phil Potter for their helpful comments on prior drafts.
Open access
