The subject of this book is the set of disciplines known as the social sciences (which in earlier times would have been referred to as the moral or human sciences). By this is meant a scientific study of human action focusing on elements of thought and behavior that are in some degree social (nonbiological). “The object of the social sciences,” writes Hans Morgenthau, “is man, not as a product of nature but as both the creature and the creator of history in and through which his individuality and freedom of choice manifest themselves.” Wherever nurture matters more than nature, or where some significant decisional element is involved, we are on the turf of social science. (This does not mean that genetic dispositions are eliminated from consideration; indeed, they comprise an active research agenda in the social sciences today. However, one presumes that any outcome of interest to the social sciences is not entirely biologically determined; there must be a significant component of choice.)

Having discussed the formal (super-empirical) criteria of a good argument, we turn now to the empirical portion of social science research, the hoped-for encounter with reality. This stage may be referred to variously as analysis, assessment, corroboration, demonstration, empirics, evaluation, methods, proof, or testing. (While acknowledging the subtle differences among these terms, I shall treat them as part of the same overall enterprise.)

Of course, the distinction between theory formation and theory-testing is never clear and bright. As is the case everywhere in social science, tasks intermingle. One cannot form an argument without considering the empirical problem of how to appraise it, and vice versa. Moreover, the task of (dis)-confirming theories is intimately conjoined with the task of forming theories. As Paul Samuelson notes, “It takes a theory to kill a theory.”

I hope that my chosen approach to social science methodology strikes readers as commonsensical. Indeed, none of the tasks, strategies, and criteria were invented by the author (though I have chosen labels for things that do not have established names), and most have received extensive discussion. From this perspective, the present book qualifies as a compendium of truisms – a function shared, I might add, by any integrative work on methodology. The first, and perhaps most important, justification for the proposed framework is that it represents a formalization of what we already know.

Nevertheless, readers are bound to have qualms about some elements of the argument. They might take issue with the criterion of generality, for example. They might like what I have to say about description, but not about causation. On what grounds might one adjudicate this sort of dispute?

Chapter 9 set forth general criteria applying to research designs whose purpose is to assess causal relationships. In this chapter and Chapter 11 I lay out specific strategies of causal inference. To be sure, there is a high degree of overlap between these topics. Indeed, each strategy can be analyzed according to the criteria that it fulfills (or does not fulfill). Yet the shift from criteria to strategies is an important one. While Chapter 9 is about principles, Chapters 10 and 11 come closer to a “how-to” guide, complete with detailed discussions of specific studies.

Strategies of causal inference will be divided into three categories: (1) randomized designs; (2) nonrandomized designs; and (3) strategies that move beyond X and Y, as summarized in Table 10.1. Because this covers a lot of ground the material is divided into two chapters, with this chapter focused on the first two topics and Chapter 11 on the third.

I argued in Part II for a resuscitation of descriptive inference within the social sciences, both as a topic of methodology and as a topic of substantive research. However, I do not suppose that description will displace causation as the reigning motif of social science. We wish to know not only what happened but also, perhaps more critically, why these things happened.

Causation is the central explanatory trope by which relationships among persons and things are established – the cement of the universe, in Hume’s much quoted words. Without some understanding of who is doing what to whom we cannot make sense of the world that we live in, we cannot hold people and institutions accountable for their actions, and we cannot act efficaciously in the world. Without a causal understanding of the world it is unlikely that we could navigate even the most mundane details of our lives, much less matters of long-term policy. This is obvious in the policy world, where causal understanding undergirds any rational intervention. And it is obvious in other areas of politics, for example, in social movements, lobbying, voting, and revolutionary change. Anyone who engages in these activities must be conscious of oneself as a causal actor in the world, and accordingly, must make assumptions (implicit or explicit) about what one’s actions might achieve – whether one supports the status quo or wishes to undermine it. Lenin, like Metternich, was vitally concerned with the causes of revolution. Even where causal understanding does not lead to social change (for not all causal analysis is directly relevant to public policy, and more to the point, not all policy proposals are implemented), we are likely to be reassured when we can order events around us into cause-and-effect relationships. “When we have such understanding,” notes Judea Pearl, “we feel ‘in control’ even if we have no practical way of controlling things.”

I cannot resist inserting a few words on the stylistic properties of social science. Although not methodological in the strict sense of the term, it is nonetheless difficult to separate the desiderata of good writing from the desiderata of good argumentation. Kristin Luker comments:

In this respect, writing shares certain characteristics in all realms in which it is employed. Good writing is good thinking, to paraphrase Orwell.

A: (Knock)

B: Come in.

A: Ah, Is this the right room for an argument?

B: I told you once.

A: No you haven’t.

B: Yes I have.

A: When?

B: Just now.

A: No you didn’t.

B: Yes I did.

A: You didn’t

B: I did!

A: You didn’t!

B: I’m telling you I did!

A: You did not!!

B: Oh, I’m sorry, just one moment. Is this a five minute argument or the full half hour?

Monty Python, "The Argument Clinic"

Argumentation in contemporary social science descends from the ancient art of rhetoric and the equally ancient science of logic. A complete argument consists of a set of key concepts, testable hypotheses (aka propositions), and perhaps a formal model or larger theoretical framework. A causal argument should also contain an explication of causal mechanisms (Chapter 8). An argument is what we speculate might be true about the world; it engages the realm of theorizing.

Sometimes, it is important to distinguish among arguments lying at different levels of abstraction. The most abstract may be referred to as macro-level theories, theoretical frameworks, or paradigms. Examples would include structural functionalism, modernization theory, exchange theory, symbolic interactionism, or conflict theory. At a slightly less abstract level one finds meso-level theories or models. And at the most specific level one speaks of hypotheses, inferences, micro-level theories, or propositions, which are assumed to be directly testable. (Explanations may apply to any level.) So, for example, work on the topic of school vouchers might include a general theory about why consumer choice enhances the educational process, a formal model incorporating various elements of that theory, and a specific hypothesis or set of hypotheses regarding the impact of a voucher-based intervention on educational attainment.

We have now reviewed randomized and nonrandomized approaches to the assignment problem, and to causal analysis more generally. Our discussion has focused on the ways in which these strategies attempt to isolate the covariational pattern between a causal factor, X, and an outcome, Y. This pattern may be of many sorts: positive or negative, proximal or distal, and so forth. There are many types of causal relationships (as summarized in Table 9.2), and each presupposes a somewhat different covariational pattern. The key point is that X/Y covariation is a necessary condition of causality. If Y does not vary with X, at least some of the time and in some real or imagined sample, then X cannot be a cause of Y. Accordingly, the discovery of X/Y covariation may provide strong evidence of causality, if various assumptions hold.

However, the problem of confounders is ubiquitous when one is dealing with nonexperimental data. (Additionally, background factors lying orthogonal to the factor of theoretical interest may present an obstacle to causal inference if the signal is overwhelmed by stochastic noise.) Consequently, one is often obliged to move beyond an exclusive focus on X and Y. Other factors will need to be measured, and conditioned, if the covariation between X and Y is to be interpreted as evidence of a causal relationship.

Broadly stated, the goal of science is to discover new things about the world and to appraise the truth-value of extant propositions about the world. Consider our exemplars, democracy and vouchers, introduced in Chapter 1. We want to uncover new things about the process of democratization and the impact of vouchers on school performance. At the same time, we want to test extant theories about these two subjects. Social science may, therefore, be understood as a twin quest for discovery and for appraisal, as summarized in Table 2.1.

The chapter begins by introducing these goals, followed by a review of their implications for more specific methodological tasks. The next section approaches the goal of discovery through the concrete task of finding a research question. Since the remaining chapters of the book assume that a research question – perhaps even a specific hypothesis – has been identified, this chapter functions as a prologue to the rest of the book.

Description will be understood in this book as any empirical argument (hypothesis, theory, etc.) about the world that claims to answer a what question (e.g., how, when, whom, or in what manner). By contrast, wherever there is an implicit or explicit claim that a factor generates variation in an outcome the argument will be regarded as causal. The distinction between these two key concepts thus hinges on the nature of the truth-claim – not on the quality of the evidence at hand, which may be strong or weak. Description is the topic of Part II, while causation is the topic of Part III. Description rightly comes first; one must describe in order to explain (causally). However, the reader will find many comparisons and contrasts across the two topics interwoven throughout the book.

Because this book is focused on generalizing statements about the world (Chapter 1), I am not concerned with descriptions that reflect only on individual cases or events (without any attempt to exemplify larger patterns). Consequently, in this book description is always an inferential act. To generalize is to infer from what we know (or think we know) to what we do not know. One sort of inferential leap is from observations within a sample that are deemed secure to those that are uncertain or missing (problems of “measurement error” or “missing data”) and to dimensions that are inherently unobservable (“latent characteristics”). Another sort of inferential leap is from a studied case or sample to a larger (unstudied) population. In both respects, descriptive models offer a “theory” about the world, “a ‘formula’ through which the data can be reproduced.”

All the tasks, strategies, and criteria noted in the previous chapters are considered valid, ceteris paribus. And yet ceteris is not always paribus. A key theme of this book is that methodological choices frequently involve tradeoffs. Satisfying one dimension may involve sacrificing another. Tasks, strategies, and criteria often conflict. Accordingly, for every dimension listed in Table 1.1 and in subsequent tables throughout the book one can locate conflicting imperatives.

The call for discovery is at odds with the call for accurate appraisal. Indeed, exploratory research is typically carried out in a different fashion than research whose primary goal is confirmatory (falsificationist), as emphasized in Chapter 2.

How do social scientists describe a social reality? What arguments do we employ in our attempts to bring order to the great blooming, buzzing confusion of the world? One might suppose that the shape of a descriptive inference is limited only by the social phenomenon that we seek to describe, the models (cognitive, linguistic, mathematical, and visual) that we have at our disposal, and our imagination. In practice, however, descriptive inferences draw from a standard itinerary of tropes.

I shall argue that most descriptive claims can be classified as indicators, syntheses, typologies, or associations, along with their various subtypes as illustrated in Table 6.1. This is how social scientists carve up nature at the descriptive level. These are the patterns that we look for when attempting to describe classes of events in the social world.