Introduction

Social scientists increasingly champion multi-method research – in particular, the use of both quantitative and qualitative tools for causal inference. Yet, what role does process tracing play in such research? I turn in this chapter to natural experiments, where process tracing can make especially useful and well-defined contributions. As I discuss, however, several lessons are relevant to other kinds of multi-method research.

With natural experiments, quantitative tools are often critical for assessing causation. Random or “as-if” random assignment to comparison groups – the definitional criterion for a natural experiment – can obviate standard concerns about confounding variables, because only the putative cause varies across the groups. Other factors are balanced by randomization, up to chance error. Simple comparisons, such as differences of means or percentages, may then validly estimate the average effect of the cause, that is, the average difference due to its presence or absence. Controlling for confounding variables is not required, and can even be harmful.

However, much more than data analysis is needed to make such research compelling. In the first place, researchers must ask the right research questions and formulate the right hypotheses; and they must create or discover research designs and gather data to test those hypotheses. Successful quantitative analysis also depends on the validity of causal models, in terms of which hypotheses are defined.

Introduction

In an agenda-setting essay first published in 2002, Lisa Martin and Beth Simmons argued that the study of international organizations (IOs) and institutions (IIs) had reached an important threshold, focusing less on why they exist and more on “whether and how they significantly impact governmental behavior and international outcomes” (Martin and Simmons 2002: 192). Put differently, the past decade has seen a sustained move by students of international institutions and organizations to viewing their subject matter as independent variables affecting state interests and policy. Conceptually, this has put a premium on identifying the mechanisms connecting institutions to states; methodologically, there has been a growing concern with measuring process.

In this chapter, I assess several studies that make claims about international institutions influencing state-level action through various processes and mechanisms. The move to process and to the method of process tracing has been salutary, I argue, producing rich and analytically rigorous studies that demonstrate the multiple roles – good and bad – played by institutions in global politics.

One of the attractions of process tracing is that it does not require technical training to be able to use the method. Just as one can be a decent cook with only a recipe and an intuitive understanding of chemistry, it is possible to do process tracing well by following the prescriptive advice in Chapter 1 and having an intuitive understanding of its logic. Yet, sometimes our intuitive understandings lead us astray. Just as a cook who understands chemistry will be better able to develop new recipes, adapt to different kitchens and ingredients, and teach cooking to others, researchers who understand the underlying logic of process tracing are likely to be better able to do it, teach it, and defend their applications of it.

This appendix thus outlines the mathematical logic of process tracing. Although technical, it should be accessible to readers with modest exposure to algebra, probability theory, and formal logic. The Appendix focuses on Bayesian reasoning as one way of illuminating the logic that underlies deductive process tracing. An important caveat here is that although the logic of process tracing and that of Bayesianism have much in common, they are not entirely coterminous. In particular, the use of process tracing to generate theories by “soaking and poking” in the evidence does not (yet) have a place in Bayesian epistemology. Also, the logic underlying process tracing can be explicated in terms of set theory and directed acyclic graphs as well as Bayesianism. Yet, Bayesianism is the inferential logic that has been developed the furthest in the context of process tracing. Accordingly, this appendix first outlines the fundamentals of Bayesianism and then advances six important implications for process tracing. The underlying premise here is that more rigorous and explicit use of inferential logic in process tracing, whether Bayesianism, directed acyclic graphs, set theory, or some other logic, will contribute to better process tracing.

Introduction

Process tracing is an invaluable tool in the civil war scholar’s toolkit. Or, rather, it should be, for it provides the ability to move beyond statistical association toward causal inference about why (and how) outcomes are produced in civil war settings. Yet, scholars have too often neglected its use. Instead, great pains have been taken to construct research designs that (at best) are able to identify suggestive correlations between variables, but lack the ability to test the mechanism(s) at work. Qualitative research is not immune to this criticism, either, for process tracing, when properly conducted, establishes a standard for rigor that often goes unmet even in detailed historical cases (see also Evangelista, this volume, Chapter 6). This is an unfortunate state of affairs; without understanding the causal processes that underpin associations, we foreclose opportunities to advance our theories of civil war and contribute to debates about the efficacy of different policies in violent settings.

This chapter emphasizes the practicalities of marrying design-based inference with the strengths of process tracing to improve our ability to build and (especially) test theories about civil war onset and dynamics. Bennett and Checkel’s ten best practices for process tracing (this volume, pp. 20–31) provide a springboard for a discussion of how to identify and conduct rigorous process tracing in settings marked by poor (or no) data, security concerns, and fluid events. The chapter also introduces ideas from the now-burgeoning literature on causal inference to help guide decisions about case selection and evidentiary standards. In particular, the approach advocated here draws on a potential outcomes framework that hinges on the use of counterfactual observations, “elaborate” theory, and qualitative evidence on treatment assignment to facilitate drawing causal inferences about why wars break out and how they are fought (see also Dunning, this volume, Chapter 8, on using process tracing to assess assignment to treatment).

Introduction

In the previous chapters, we used a regression-based approach for case selection. Like any approach, relying on regression has important trade-offs that researchers need to be aware of. Perhaps most importantly, this approach relies on the functional form assumptions of the underlying regression analysis, which researchers may not want to adopt in selecting cases for pathway analysis. The good news is that there is nothing about pathway analysis that requires the use of regression analysis. We have concentrated on regression analysis because it is well known by researchers, and we think likely to be used in practice. We recognize that there are a host of other options to leverage existing large-N data to select cases for comparative pathway analysis.

This chapter illustrates one alternative to regression-based case selection. It explores how a researcher might use a common matching approach to select cases, and we apply the technique to an example we also discuss in the next chapter: the relationship between natural resource exports and civil conflict. A matching approach is useful, because it is explicitly designed for making comparisons. This is valuable in our context, because it fits well with the underlying importance of comparative research strategies. It is worth noting that others have also combined matching approaches with case studies, but the goal in prior approaches has been to understand causal effects (Nielsen 2014; Rosenbaum and Silber 2001); in general, matching approaches aimed at improving causal inference focus on identifying comparable treatment and control cases based on pretreatment co-variates that are related to treatment (Ho et al. 2007; Rubin 2008). In pathway analysis, however, the goal differs. By definition, pathway analysis assumes that the causal effect has been identified in previous scholarship and matching is used as a means to identify cases that are appropriate for exploring the mechanisms underlying the previously identified relationship.

Introduction

So far in this book we have focused on using various quantitative methods to guide case selection in order to explore the substantive relationship between a key explanatory variable (X1) and an outcome (Y). The discussion assumes that, in some settings, researchers do not have strong reasons for selecting particular cases, and so are using quantitative data to determine case selection. However, in other cases, researchers often have compelling reasons for examining certain cases, including theoretical, empirical, and practical reasons (such as access to data sources, cost effectiveness, and preexisting expertise). Under these circumstances, the issue is not which cases to choose initially, but how to gain perspective on the cases already chosen and perhaps how to choose cases for additional analysis.

This chapter illustrates how large-N data can help gain perspective on previously chosen cases by shedding light on how they fit within the broader universe of cases that could have been chosen, thus providing some leverage over how the extant case studies compare to unstudied cases. This can help a researcher understand how to generalize to unstudied cases and how to identify future cases that might offer promising comparisons.

The use of large-N data to gain perspective on previously selected cases is best explored by example. In this chapter we focus on the relationship between natural resources (X1) and civil conflict (Y), which was also the focus of Chapter 6.We first explain why this example is useful, then we review the existing pathway analyses conducted by Michael Ross (2004) and the large-N data analysis of Paul Collier and Anke Hoeffler (2004), and finally we demonstrate how to use the large-N data to gain some perspective on the cases selected by Ross and how to identify avenues for future inquiry.

Introduction

The primary purpose of pathway analysis is to build knowledge about the causal mechanisms that link X1 to Y across settings. This implies the need to perform comparative analyses that can generate knowledge and/or hypotheses about the broader population of cases that feature the X1/Y relationship. Before delving into the details of case selection, it is important to consider how researchers should prepare for pathway analysis. As an initial matter, researchers must clarify their specific goals. Once oriented to the basic task, they should: (1) determine whether the basic analytic requisites of pathway analysis are met; (2) identify what is already known about the X1/Y relationship; and (3) take stock of relevant measures.

The threshold task: clarifying the goals of pathway analysis

The purpose of using pathway analysis differs from other, more familiar types of research, which is why it is important for researchers to clarify these goals from the start. On this score, James Mahoney and Gary Goertz (2006) usefully distinguish between two ideal types of research in the social sciences: causes of effects and effects of causes. Causes-of-effects research seeks to provide “thick description” of the emergence of singular events or outcomes in particular settings (Mahoney and Goertz 2006; see Brady and Collier 2004; Geertz 1973). For US politics or history research, for example, possible questions might include: Why World War II? Why did the New Deal coalition collapse? What factors caused the most recent financial crisis? Why did the US Supreme Court decide Bush v. Gore as it did? For these types of questions, case selection is driven mostly by the substantive importance of the outcome to be explained, and the research primarily emphasizes case-specific internal validity (“Did the research get the story right?”), and not external validity (“Do the story's lessons apply in other contexts?”), although scholars often try to draw some broader lessons from their detailed descriptions.

Introduction

In the last chapter, we applied our method to several simple textbook examples, including a hypothetical that posited a linear X1/Y relationship with equifinality (that is, multiple pathways between X1 and Y). In this chapter, we turn to a discussion of case selection in a more complex empirical setting: one that features a non-linear relationship between X1 and Y and the relationship is embedded in a longer causal chain that involves dynamic processes. Under these conditions, researchers must decide where in the causal process to investigate and how to deal with temporal dynamics in their large- and small-N studies. This chapter grapples with these issues by using an example drawn from the policy diffusion literature. As we make clear, in the context of pathway analysis, conceptualizing what is the key explanatory variable and what is the mechanism is not determined by the labels used in prior research or deep philosophical concerns, but rather by practical concerns regarding the relevant research question, how variables are used within the existing empirical research, and data availability. Once the relevant literature is properly understood, our method can be applied in a relatively straightforward manner.

International policy diffusion

The literature on international policy diffusion is both immense and contested. For purposes of this chapter, therefore, we distinguish between two very different types of studies. One type focuses on the threshold question of whether policy choices spread across political boundaries; that is, whether policy adoption is purely a function of domestic pressures (Berry and Berry 1990; Gray 1973, 1994; Simmons et al. 2008; cf. Volden et al. 2008). The second type of study essentially assumes that policies diffuse across borders, so its focus is on how and under what conditions policy decisions of one country spill over into others. In general, the literature that assumes the existence of diffusion and focuses on how policy diffusion works is more interesting for our purposes, as the search for mechanisms lies at its center (Simmons et al. 2008).

Introduction

To this point, we have mainly focused on the early stages of mechanism-centered research, particularly selecting cases for pathway analysis that explore the links underlying an association between some explanatory variable (X1) and an outcome (Y). However, the results of pathway analysis not only can provide a map of mechanisms underlying an X1/Y relationship, but also can be used to “look forward” to provide insight into the feasibility and nature of future studies of mechanisms. From this vantage, pathway analysis is best understood as part of a larger research agenda that spans multiple years, multiple research approaches, and multiple researchers.

The forward-looking purposes of pathway analysis are largely ignored in the literature for a variety of possible reasons. Partly this omission may reflect the limited state of knowledge about mechanisms in many areas. Quantitative studies of mechanisms require that a researcher has already identified some basic information about the mechanisms linking X1 and Y and understand the basic structure of the X1/Y relationship (see Table 2.2 in Chapter 2). This omission also may reflect that the literature on how to study mechanisms quantitatively is relatively new and rapidly evolving. Nevertheless, there are some areas in which researchers have accumulated a considerable amount of insight into mechanisms, and there are compelling reasons for pursuing further mechanism-centered research. For researchers working in such an area, the question to ask is: How might insights from pathway analysis inform future studies on mechanisms?

Putting the pieces together

This book has examined pathway analysis from a number of angles: how to prepare for it by reading the literature in light of its analytic requisites and different ideal types of X1/Y relationships; how to select cases for it based on the expected X1/Y relationships and variation in case characteristics; how to use these tools to gain perspective on cases already selected for process tracing; and how the results from pathway analysis might inform future studies of mechanisms. This chapter seeks to put these pieces together and review the role of pathway analysis in a continuing mixed-method research agenda on mechanisms. The argument is that pathway analysis serves as a critical bridge from what we know about an X1/Y relationship through large-N studies to detailed maps of the mechanisms connecting X1 and Y, which can be used to assess the feasibility of future quantitative studies of mechanisms and the appropriate goals of future qualitative work on mechanisms. From this perspective, good pathway analysis advances the search for mechanisms by systematically building on what we already know about the X1/Y relationship, generating insights into the links between variables, and clarifying avenues of future inquiry.

The role of pathway analysis in the mixed-methods search for mechanisms

A central theme of the book is that pathway analysis constitutes a distinct mode of inquiry. Whereas most research in the social sciences seeks to understand the causes of events or estimate average effects of some variable, X1, on an outcome, Y (Mahoney and Goertz 2006), pathway analysis seeks to (1) understand the mechanisms underlying the X1/Y relationship in particular cases and (2) generate insights from these cases about mechanisms in the unstudied population of cases featuring the X1/Y relationship. By its very nature, pathway analysis connects the literature on the X1/Y relationship to an ongoing, mixed-method research agenda that first seeks to map mechanisms so that we have a better understanding of the X1/Y relationship and then seeks to inform future, mechanism-centered work.