In most social psychological studies, researchers conduct analyses that treat participants as a random effect. This means that inferential statistics about the effects of manipulated variables address the question whether one can generalize effects from the sample of participants included in the research to other participants that might have been used. In many research domains, experiments actually involve multiple random variables (e.g., stimuli or items to which participants respond, experimental accomplices, interacting partners, groups). If analyses in these studies treat participants as the only random factor, then conclusions cannot be generalized to other stimuli, items, accomplices, partners, or groups. What are required are mixed models that allow multiple random factors. For studies with single experimental manipulations, we consider alternative designs with multiple random factors, analytic models, and power considerations. Additionally, we discuss how random factors that vary between studies, rather than within them, may induce effect size heterogeneity, with implications for power and the conduct of replication studies.

The conclusions close the manuscript and make four points. First, they review the macro-level observational expectations tested in Part II, and how my findings, obtained through a triangulation of different techniques, allow for a comprehensive picture of how war affected state formation throughout the entire region. Second, they bring together all case studies in Part III, noting how the historical evidence collected fits the expectations of the theory at a micro-level—e.g., considering the behavior of individual actors and the effects of narrow events like battles within wars—and does so with out-and-out consistency—i.e., case by case, almost without exception. Third, they reflect upon the scope of the theory, discussing many other cases that could be explained by the long-term effects of war outcomes. This discussion covers many regions and time periods, showing that classical bellicist theory not only can travel, but can also solves logical problems and empirical puzzles highlighted by previous scholarship. Finally, the conclusions suggest many lines of enquiry for future research that the book leaves open.

Inductive reasoning involves using existing knowledge to make predictions about novel cases. This chapter reviews and evaluates computational models of this fundamental aspect of cognition, with a focus on work involving property induction. The review includes early induction models such as similarity coverage, and the feature-based induction model, as well as a detailed coverage of more recent Bayesian and connectionist approaches. Each model is examined against benchmark empirical phenomena. Model limitations are also identified. The chapter highlights the major advances that have been made in our understanding of the mechanisms that drive induction, as well as identifying challenges for future modeling. These include accounting for individual and developmental differences and applying induction models to explain other forms of reasoning.

This chapter illustrates how to apply Bayesian reasoning when analyzing more than one case. The same principles that govern Bayesian updating with multiple pieces of evidence apply to Bayesian inference with multiple cases.

We reflect on the relative ‘success’ versus ‘failure’ of psychology as a research field, and we challenge the widelybheld notion that we are in a reproducibility (or replication) crisis. At the centre of our discussion is the question: does psychology have a future, qua science, if the phenomena it studies are changing all the time and contingent on fleeting contexts or historical conditions? This chapter describes how there is only a reproducibility crisis if we adopt assumptions and expectations that enact a substance ontology. In contrast, we describe how variability is to be expected if we adopt a process ontology. We argue that the way out of the current ‘crisis’ is therefore not necessarily more methodological and experimental rigour, but a fundamental shift in what we should expect from psychological phenomena. We call for a prioritization of understanding the ways in which phenomena are socially situated and context-contingent, rather than an unrealistic need to replicate.

This chapter discusses the practice of measurement in psychological research. Here, where we cast doubt on the basic assumptions and endeavours underlying the act of measuring in mainstream psychology. Next, we introduce the processual alternative, which stresses the study of activity as situated and coupled with an environment. This chapter explains how a process approach to ‘measurement’ is thus fundamentally different from the standard one, and can remedy existing issues related to non-ergodicity and the ecological fallacy. These ideas are illustrated by means of the concept of intelligence, which is undoubtedly one of psychology’s show-pieces of measurement.

Woolcock focuses on the utility of qualitative case studies for addressing the decision-maker’s perennial external validity concern: What works there may not work here. He asks how to generate the facts that are important in determining whether an intervention can be scaled and replicated in a given setting. He focuses our attention on three categories: 1) causal density, 2) implementation capability, and 3) reasoned expectations about what can be achieved by when. Experiments are helpful for sorting out causally simple outcomes like the impact of deworming, but they are less insightful when there are many causal pathways, feedback loops, and exogenous influences. Nor do they help sort out the effect of mandate, management capacity, and supply chains, or the way results will materialize – whether some will materialize before others or increase or dissipate over time. Analytic case studies, Woolcock argues, are the main method available for assessing the generalizability of any given intervention.

Machine-learning algorithms can be viewed as stochastic transformations that map training data to hypotheses. Following Bousquet and Elisseeff, we say such an algorithm is stable if its output does not depend too much on any individual training example. Since stability is closely connected to generalization capabilities of learning algorithms, it is of interest to obtain sharp quantitative estimates on the generalization bias of machine-learning algorithms in terms of their stability properties. We describe several information-theoretic measures of algorithmic stability and illustrate their use for upper-bounding the generalization bias of learning algorithms. Specifically, we relate the expected generalization error of a learning algorithm to several information-theoretic quantities that capture the statistical dependence between the training data and the hypothesis. These include mutual information and erasure mutual information, and their counterparts induced by the total variation distance. We illustrate the general theory through examples, including the Gibbs algorithm and differentially private algorithms, and discuss strategies for controlling the generalization error.

A grand challenge in representation learning is the development of computational algorithms that learn the explanatory factors of variation behind high-dimensional data. Representation models (encoders) are often determined for optimizing performance on training data when the real objective is to generalize well to other (unseen) data. This chapter provides an overview of fundamental concepts in statistical learning theory and the information-bottleneck principle. This serves as a mathematical basis for the technical results, in which an upper bound to the generalization gap corresponding to the cross-entropy risk is given. When this penalty term times a suitable multiplier and the cross-entropy empirical risk are minimized jointly, the problem is equivalent to optimizing the information-bottleneck objective with respect to the empirical data distribution. This result provides an interesting connection between mutual information and generalization, and helps to explain why noise injection during the training phase can improve the generalization ability of encoder models and enforce invariances in the resulting representations.

In this chapter, there are two types of probabilities that can be estimated: empirical probability and theoretical probability. Empirical probability is calculated by conducting a number of trials and finding the proportion that resulted in each outcome. Theoretical probability is calculated by dividing the number of methods of obtaining an outcome by the total number of possible outcomes. Adding together the probabilities of two different events will produce the probability that either one will occur. Multiplying the probabilities of two events together will produce the probability that both will occur at the same time or in succession. As the number of trials increases, the empirical probability and theoretical probability converge.

It is possible to build a histogram of empirical or theoretical probabilities. As the number of trials increases, the empirical and theoretical probability distributions converge. If an outcome is produced by adding together (or averaging) the results of events, the probability distribution is normally distributed. Because of this, it is possible to make inferences about the population based on sample data – a process called generalization. The mean of sample means converges to the population mean, and the standard deviation of means (the standard error) converges on the value.

Philosophers speculated that assocations might be crucial in guiding behavior, but it was Pavlov who first explored this question experimentally. He discovered that dogs would begin to salivate to stimuli that preceded the delivery of food; the greater the contiguity of these stimuli with food, the stronger the conditioning. He also discovered fundamental properties of conditioning including extinction, inhibition, second-order conditioning, counterconditioning, discrimination, and generalization. Watson and Raynor extended his research by showing that conditioning also occurred in humans; they showed that fear could be conditioned in an infant who became famous as Little Albert. Subsequent research used random and unpaired control groups to determine if a change in behavior was truly due to conditioning, or to sensitization or pseudoconditioning. Three responses proved particularly useful in this later research: fear conditioning (CER) and taste-aversion learning in rats, and the galvanic skin response (GSR) in humans.