Having surveyed the craft of writing, we turn to the craft of speaking, the oral form of language. Good speaking is somewhat different from good writing. Of course, one can simply read a prepared text, in which case the difference virtually disappears. But this does not generally qualify as good speaking in an academic or professional context – unless, that is, it is carefully honed to appear as if it is extemporaneous.

People expect a “live” performance. Your presence should be real, unmediated. Naturally, it may be mediated by various technologies and it may be pre-recorded; but it should feel as if it is happening right here and right now. This is the dynamic quality of public speaking. It is a special quality that is probably hard-wired in our brains and therefore carries a resonance that cannot be simulated with prose.

Public speaking is inextricably linked to comportment, i.e., how you carry yourself. Every time you make a public intervention you convey a vision of yourself. It is this persona that people tend to remember. Thus, when we say “speech” in this chapter we intend to include all the visual cues that accompany speech – dress, gaze, posture, gestures, and so forth.

In ancient times, speech was the preeminent art of persuasion. Rhetoric meant speech, and only secondarily prose or poetry. That prioritization is easy to understand in the context of a predominantly oral culture.

Nowadays, the craft of public speaking has fallen into desuetude (though some writing courses also include a component devoted to public speaking). People still talk, but speech is no longer cultivated as a professional activity, with a few exceptions such as moot courts in law school. This is unfortunate because speech is no less important today than it was a century ago. Perhaps it is destined to become more important over time as online lectures and YouTube videos replace written texts, and video calls and video conferences replace email. The spoken word may turn out to be mightier than the written word.

In any case, whether you are a good speaker or a poor speaker is likely to affect how successfully you can get your ideas across – not to mention getting good grades, landing a job, and succeeding in your chosen profession. To be sure, college courses generally don't allocate much credit for participation.

Recall (from Chapter 6) that estimating a causal effect involves comparing a factual (that which actually happened) to a counterfactual (that which might have happened). Returning to our perennial example, let us say that we wish to estimate the effect of a job-training program on the earnings of unemployed people after they have completed the program. Those who participate in the program are members of the treatment group. Non-participants are members of the control group.

Let us say that we know the earnings of participants and non-participants after completing the program. These are the factuals. What we do not know is what their earnings would have been if their roles had been reversed. These are the counterfactuals.

Although the only sure way to compare factuals with counterfactuals is to employ a time-machine, a well-designed experiment comes close to the mythical time-machine insofar as the control group exemplifies the (unobserved) counterfactual for the treatment group, and the treatment group exemplifies the (unobserved) counterfactual for the control group. Under certain conditions, this is a plausible scenario.

For present purposes, the defining criterion of an experiment is that the treatment is randomly assigned (“randomized”) across subjects. We do not care who controls the experiment – the researcher conducting the study or someone else. (Sometimes, an experiment that unfolds naturally, without intervention by a researcher, is referred to as a natural experiment, as discussed below.)

We begin this chapter with a review of the problem of confounding as it applies to experimental research. We proceed to introduce various approaches to experimental research. The final section offers a series of examples of experiments conducted on a variety of diverse topics. Together, these sections should give the reader a sense of the opportunities for, and limitations of, experimental research in the social sciences.

Experiments With and Without Confounding

In its simplest version, a single treatment (e.g., a worker-training program) is randomly assigned to members drawn from a known population (e.g., unemployed people of a given age). That is, some are chosen to participate in the program (the treatment group) and others are chosen to become part of the control group. The causal effect of the program is measured by comparing annual earnings of program participants with annual earnings of non-participants one year after the completion of the program.

Once upon a time, the practice of social science could be understood as the application of commonsense and intuition – something you might develop in the course of growing up. This is no longer true, or only partly true. Although commonsense and intuition are still useful, the social science disciplines have moved well beyond what can be understood without specialized training.

If you want to become an artist, musician, engineer – or pretty much anything, these days – developing your technique in these highly specialized areas is essential. It takes great dedication, countless hours of concentrated work, and professional guidance. The same may be said for social science in the contemporary era. One may mourn the death of the amateur social scientist. But one might as well reconcile oneself to the fact.

In response, methods courses have proliferated at both the undergraduate and graduate level. Likewise, methodological skills are in high demand in the social sciences and their cognate professions. Successful careers in government, communications, education, social work, business, law, and all of the policy fields require a solid grounding in methodology. Whether one is applying for graduate programs or for a job, the material covered in this book should stand one in good stead.

Indeed, a working knowledge of social science tools of analysis may prove more crucial for one's career than whatever substantive knowledge one acquires in the course of a college education. What one knows is less important than what one can do, and what one can do depends on a working knowledge of methodology.

These developments may be viewed as part of a broader sea-change, driven by the rise of computers and the Internet. With sophisticated IT tools at our disposal, factual knowledge about a subject is no longer at a premium and can usually be obtained from a Google search or from a specially designed database in milliseconds. Likewise, any repetitive procedure can be programmed as a set of algorithms on a computer. This means that the value of an education is no longer in the facts or established protocols you might learn. This sort of knowledge can be produced by machines in a more timely and accurate fashion than by the human brain.

Statistics can be more than a tool for describing data. In the social sciences we have hypotheses that move us beyond simple descriptions of populations to relationships between two or more variables. To analyze these relationships we often rely in practice on statistical inference. That is, we need to make decisions based on data collected on a small group (sample) for the larger group that we want to study (population). In Chapter 4 we introduced the concept of sampling and provided a description of the three general objectives pertaining to sampling: representativeness, size, and level of analysis. Here we move to a more focused explanation of how sampling helps researchers overcome practical limitations and what implications this has for quantitative analyses.

Sampling

As we noted earlier, when it comes to sampling, larger is generally better, the aforementioned issues aside. So why sample at all? Why not collect data on the full population? In all types of research we find the same two answers: time and money. Research occurs in the real world and collecting the ideal data is often limited by how much time and money the researcher can direct to the project. Thus every research design must take into account the practical circumstances. In doing so, researchers necessarily restrict their investigation into a sample or subset of the population that they would like to study.

In general, when we make comparisons we would like to talk about more than just the observations in the sample. We would like to talk about the larger group of interest in our research, the population. How we do this is the objective of statistical inference. More specifically, statistical inference helps researchers provide statements of confidence in our ability to generalize or infer from the sample to the population. The ability to offer an estimate of relative precision is another reason why quantitative empirical research is so useful and popular.

Consider the ANES dataset that we used for the examples in Chapter 18. How many cases or individuals do we have in our sample? Obviously, the nearly 6,000 individuals in that sample is not anywhere near the size of the full voting age population of the United States, yet we would like to use this sample of data to describe that population.

This part of the book is focused on fundamental elements of social science, elements that form building blocks for everything else. In Chapter 1, we lay out the rationale for a unified approach to our subject, social science methodology. In Chapter 2, we discuss social science arguments, with primary attention to descriptive and causal arguments. In Chapter 3, we turn to the topic of conceptualization and measurement. In Chapter 4, we discuss the generic features of empirical analysis.

The first step toward analysis is to get your data into a format that is convenient for the sort of analysis you wish to pursue. This, in turn, probably depends on the sort of data you are collecting. Here, we shall distinguish among four data types: qualitative, medium-N, large-N, and textual (though one can have data that is a combination of these; e.g., medium-N and qualitative, large-N and textual … etc.). Methods for handling these data types are continually invented and reinvented, so the reader may wish to consult other sources to obtain the most up-to-date information on these subjects. Our intent is to provide a useful overview, in any case, not to delve into the details.

Qualitative Data

Suppose that you are trying to integrate data drawn from a limited number of units with a diversity of evidence. The evidence may have been gathered with any of the techniques (or combination of techniques) discussed in Chapter 13. It might include text, photos, maps, and other media. The nature of the material might be variegated – a combination of what informants said and did, the researcher's own observations and theories, multi-media artifacts, locations, relevant articles from academic journals and/or popular media, feedback from colleagues, and so forth. Some of the evidence may apply across all studied units in the sample while some is specific to certain units. But one doesn't have systematic observations for a limited set of variables across all units in the sample. It may not even be clear what the variables are, what the sample is, or what the population of the study is. There may be – at least initially – no specific hypothesis but rather a general research question that awaits further refinement. In other words, the investigation may be more exploratory (to discover a theory or hypothesis) than confirmatory (to test a theory or hypothesis).

This setting exemplifies a good deal of work often described as qualitative, so we shall refer to it as qualitative data (as defined in Chapter 4).

Social science is organized around arguments. This is what we hope to prove, or disprove, by an empirical analysis. Science proceeds as arguments are advanced, revised, and proven or disproven.

A theory is the reasoning behind an argument. It explains a relationship, its mechanisms (if the relationship is causal), scope-conditions, background conditions, and any additional information needed to interpret the argument. One might theorize that democratization enhances economic growth because it holds leaders accountable, and accountable leaders are more likely to adopt policies that serve the public good.

If a theory is presented in a formal manner, perhaps with a set of mathematical expressions, it may be referred to as a model. At present, however, we do not distinguish between an argument, theory, or model; these terms will be used more or less synonymously.

A hypothesis is the specific, testable element(s) of any argument. One might hypothesize that as a country becomes more democratic its GDP (gross domestic product) growth increases. Our use of the term “argument” is thus meant to encompass both theory and hypothesis.

This chapter begins by distinguishing various types of social science argumentation. Arguments central to social science are generally classifiable as either descriptive or causal, so we spend considerable time on these genres. Next, we contrast these genres with other sorts of arguments such as those that are predictive, normative, or prescriptive. Finally, we lay out the characteristics of a good descriptive or causal argument. (Additional criteria specific to causal arguments are postponed until Part II of the book.)

Descriptive Arguments

A descriptive argument describes some aspect of the world. In doing so, it aims to answer what questions (e.g., when, whom, out of what, in what manner) about a phenomenon. Descriptive arguments are about what is or what was.

Note that many features of the world are intrinsically important. We want to know which countries are democratic and which are not, and in what ways they are democratic or undemocratic. We want to know how many Jews, gypsies, gays, and socialists were killed in the European Holocaust. We want to know the intellectual origins of individualism, and how it evolved through time and across cultures. We want to know whether the media in a country presents a biased view of news events, and in what direction that bias runs.

Ethics refers to formal and informal codes of behavior, i.e., the rightness or wrongness of an action as judged by a community. In the present context, we are of course concerned with ethical norms that apply to the activity of social science.

Ethical concerns are often raised in conjunction with experiments, where some subjects are assigned to a treatment condition and others to a control condition. If the treatment condition is potentially dangerous – or potentially beneficial – there are obvious ethical concerns. Why should one group of subjects be singled out in a wholly arbitrary fashion? What could justify such an intervention?

While they may be less apparent, ethical concerns also impinge upon other aspects of social science, e.g., to the collection of data from human subjects by observational methods, data analysis, and the dissemination of data and results. Arguably, everything that one might do, or refrain from doing, has ethical implications. Consequently, we take a broad view of the subject.

The chapter begins by discussing the juridical conception of ethics – centered on statutes, legal precedents, and institutional rules. Next, we discuss the central importance of protecting human subjects. Finally, we explore the activity of social science as an ethical sphere of action. What is right (moral) conduct for a social scientist?

Juridical Ethics

The juridical conception of ethics refers to rules one is obliged to follow as a member of a university, a profession, or a country. Universities in the United States are obliged, as a condition of receiving federal funding, to establish a set of procedures to govern research with human subjects. This involves the formation of an Institutional Review Board (IRB) to apply the protocol and to approve specific research proposals submitted by students or university employees. (IRBs may also be referred to as ethics review boards.) All research on human subjects is subject to approval by the IRB. Even research that does not involve human subjects – such as historical research – may require IRB exemption. Thus, IRBs have become an integral part of scientific research at all American universities – at least, all universities that receive federal funding.

In the previous chapter we discussed general causal frameworks, the building blocks of a causal explanation. In this chapter, we focus on specific hypotheses, where one factor is thought to generate change in another factor.

We begin by clarifying the concept of causality. In the next section, we discuss the criteria of a good (well-constructed) causal hypothesis. The rest of the chapter is devoted to causal analysis. First, we outline the criteria that all causal research designs seek to achieve. Next, we discuss the problem of reaching causal inference.

This chapter is fairly complex. A number of new terms are introduced, some of which may be unfamiliar to the reader and some of which are used in slightly different ways in different disciplines. Although the vocabulary may seem bewildering at first, try to familiarize yourself with these concepts – which you are likely to encounter in your reading and in your future work. The topics covered here are critical for understanding how evidence is used to infer causality in social-science settings. Whether you are primarily a consumer or a producer of social science the following chapters bear a close read and a good think.

Causality

A causal hypothesis involves at least two elements: a cause and an outcome. A cause may be referred to variously as a condition, covariate, exogenous variable, explanatory variable, explanans, independent variable, input, intervention, parent, predictor, right-side variable, treatment, or simply “X.” An outcome may be referred to as a dependent variable, descendant, effect, endogenous variable, explanandum, left-side variable, output, response, or “Y.”

Whatever the terminology, to say that a factor, X, is a cause of an outcome, Y, is to say that a change in X generates a change in Y relative to what Y would otherwise be (the counterfactual condition), given certain background conditions (ceteris paribus assumptions) and scope-conditions (the population of the inference).

Now, let's unpack things a bit. As an example, we shall focus on the causal role of a worker-training program. A reasonable hypothesis is that participation in the program (X) will enhance an unemployed person's subsequent earnings (Y). If the relationship is causal, her earnings should be higher than they would be if she had never participated in the program. Let us represent the treatment, X, as a binary (dichotomous) variable, which takes one of two values.

The previous chapters have presented a largely non-mathematical approach to methodology. This follows our belief that most methodological issues are better communicated through prose than through math, at least at introductory levels. When methodology is conflated with statistics, as it is in some textbooks, the treatment leaves many questions unaddressed. Worse, it may cause the reader to ignore fundamental problems of inference—confusing p-values with a demonstration of causality or with social relevance, for instance.

Having laid a foundation for social science methodology we turn now to a mathematical consideration of some of these same issues. Issues introduced fleetingly in the previous pages such as sampling, statistical significance, and confidence intervals will now be given a more complete treatment. Moreover, understanding the math behind a concept makes it more apprehensible, less mysterious. Additionally, we present practical tools for analyzing data, covering both descriptive statistics and inferential statistics. The purpose of this section of the book is thus to enrich your understanding of social science and to expand your toolbox of techniques.

Because statistics is hardly separate from methodology (at large) readers will note lots of overlap between this section and previous sections of the book. Even so, there is a mathematical logic to statistics and for this reason the material is best presented in a tightly sequenced manner, separate from previous sections. The reason that we present statistics last, and not first, is that we view statistics as a handmaiden of social science methodology—not the other way around. Researchers should begin with concepts, theories, and research designs and then start discussing samples and the varying utility of different statistical models.

Some might take the view that statistics is such a vast subject that it cannot be summarized in a few short chapters. We agree that this is a non-trivial task. Others may have no need for statistics at this early stage of their social science education; and thus may set aside Part IV for later. For those who read on, however, what follows is intended to serve as a step-by-step introduction to the most commonly applied statistics in the social sciences, a mere jumping-off point for further study and other works. Ultimately, understanding the techniques and topics introduced below is a prerequisite to successfully designing quantitative research in the social sciences.

In this section of the book we consider the process of research, as it is conducted, and the presentation of research, once it has been conducted. Chapter 11 discusses how to read works of social science (i.e., books and articles produced for an academic audience). Chapter 12 offers advice on how to find a topic for a research paper. Chapter 13 lays out various strategies for gathering data. (Since this is a large topic this is an especially long chapter.) Chapter 14 discusses the task of writing. Chapter 15 addresses the task of public speaking. Chapter 16 offers a short discussion of ethical issues associated with the research process.

As you can probably appreciate, these tasks are closely interwoven. Reading social science will help you develop a topic for your research and will help you develop a sense for how to construct your own project. Likewise, writing social science will help you appreciate the strengths, and realize the weaknesses, of other studies. Public speaking is another form of communication, closely linked to writing. Sometimes, a good way to write is to express an idea out loud (or sotto voce); sometimes, it is best to write it out first. There are lots of synergies. So these chapters – and their associated activities – are best approached together.

Sometimes, the task of writing is regarded as separate and independent from the study of social science. In this view, writing is a skill taught in English, rhetoric, or composition courses while theorizing and analyzing are reserved for courses in social science.

People who hold this view may also regard writing as an art (and hence the province of the humanities) in contrast to the task of analysis, which they may regard as more scientific (and hence the province of the social sciences). They may even be a bit suspicious of eloquent prose, regarding it as a sign of sophistry (a subtle, superficially plausible, but generally fallacious method of reasoning). From this perspective, the cultivation of style is a substitute for sound analysis – “mere rhetoric,” as the phrase goes.

We can all agree that substance should trump style in the realm of social science. Yet, in order to communicate ideas must be put into words. If an author is unable to do so the reader must put the pieces of an argument together him- or herself, with much effort and possible misunderstanding. A poorly written study is one that is poorly executed. It will have less impact on the world, and – if it has any impact at all – may have an impact that is different from what the author intended. Communication skills thus matter a great deal to the progress of social science.

Moreover, the act of communicating is impossible to separate from the act of reasoning. We think through prose. This recalls E. M. Forster's question: “How can I tell what I think till I see what I say?” In our own experience, writing out an argument usually leads to a reconceptualization of that argument. One cannot express an idea cogently without first understanding that idea, and one cannot understand an idea until one has effectively expressed it. Bad writing is usually a symptom of a deeper malaise.

Of course, everything depends upon how one chooses to define “good writing.” Some purely stylistic components of writing such as spelling, grammar (norms of usage), and word-choice are indeed rather superficial in nature and do not necessarily reflect on the logic of an argument or the thoughtfulness of the writer.

Causation is the central explanatory trope by which relationships among persons and things are established – the cement of the universe, in Hume's words. Without some understanding of who is doing what to whom we cannot make sense of the world that we live in, we cannot hold people and institutions accountable for their actions, and we cannot act efficaciously in the world. Without a causal understanding of the world it is unlikely that we could navigate even the most mundane details of our lives, much less plan for the future. This is obvious in the policy world, where causal understanding undergirds any rational intervention. One must have some sense of what impact a policy is likely to have in order to support its adoption.

Even where causal understanding does not relate to future changes in the status quo we are likely to be reassured when we can order events around us into cause-and-effect relationships. “When we have such understanding,” notes Judea Pearl, “we feel ‘in control’ even if we have no practical way of controlling things.” Causality is not just a methodological preoccupation. It is also a way of relating to the world. That said, there are important differences between causal inference in everyday contexts and in social-scientific contexts.

Chapter 5 introduces a variety of causal frameworks that are widely employed in social science today. They may be viewed as the building blocks of a causal explanation. Chapter 6 defines the topic of causality, and lays out the attributes of a good causal hypothesis and the core components of causal analysis. Chapter 7 discusses experimental research designs, where the causal factor of interest is randomized across groups. Chapter 8 discusses large-N observational designs (i.e., non-experimental designs). Chapter 9 deals with case study designs, where the number of units is limited to one or several. Chapter 10 reviews and reflects on various aspects of causal inference, serving as a coda for this part of the book.

Arguments are articulated with the use of key concepts. Indeed, the argument of a study is inseparable from its key concepts since the latter are the linguistic tools with which an argument is formulated. Any study of democracy, for example, must wrestle with the problem of how to define this key term – which will guide our discussion in this chapter.

Concepts, in turn, receive empirical meaning through the indicators chosen to measure them. Any study of democracy must be concerned not only with how to define democracy but also with how to operationalize (measure) this abstract concept.

Conceptualization and measurement are thus closely linked. This is why we have chosen to present them together in this chapter, which begins with concept formation and proceeds to measurement.

Concepts

The key concepts of social science are never fixed and, regrettably, not always clear. Many abstract concepts – such as democracy or social capital – are employed in a variety of ways and thus mean different things in different contexts. This is true even of more specific concepts such as worker-training programs. (Does a one-day program focusing on advice for job-hunting qualify? How about a person who enlists government support to take classes at a community college, or an apprenticeship program?)

The persistent ambiguity of key concepts makes it difficult for the reader, who may struggle to figure out what a term means in a given context and how it connects with other work (using the same or similar terms). It also makes it difficult for writers, who must identify which of several terms they should adopt in their own work and how they should define the chosen term.

Sometimes, the task of forming concepts seems highly arbitrary. And this, in turn, may prompt readers to adopt a skeptical attitude toward the subject. At the same time, the choice of concepts is never entirely arbitrary. Some choices are usually better than others, and a few are patently absurd.

In this spirit, we offer the following criteria, intended to guide the process of concept formation in the social sciences. A good concept, we shall argue, is resonant, internally coherent, externally differentiated, theoretically useful, and consistent in meaning, as summarized in Table 3.1.