We introduce multilevel multivariate meta-analysis methodology designed to account for the complexity of contemporary psychological research data. Our methodology directly models the observations from a set of studies in a manner that accounts for the variation and covariation induced by the facts that observations differ in their dependent measures and moderators and are nested within, for example, papers, studies, groups of subjects, and study conditions. Our methodology is motivated by data from papers and studies of the choice overload hypothesis. It more fully accounts for the complexity of choice overload data relative to two prior meta-analyses and thus provides richer insight. In particular, it shows that choice overload varies substantially as a function of the six dependent measures and four moderators examined in the domain and that there are potentially interesting and theoretically important interactions among them. It also shows that the various dependent measures have differing levels of variation and that levels up to and including the highest (i.e., the fifth, or paper, level) are necessary to capture the variation and covariation induced by the nesting structure. Our results have substantial implications for future studies of choice overload.

We examine the ability of eye movement data to help understand the determinants of decision-making over risky prospects. We start with structural models of choice under risk, and use that structure to inform what we identify from the use of process data in addition to choice data. We find that information on eye movements does significantly affect the extent and nature of probability weighting behavior. Our structural model allows us to show the pathway of the effect, rather than simply identifying a reduced form effect. This insight should be of importance for the normative design of choice mechanisms for risky products. We also show that decision-response duration is no substitute for the richer information provided by eye-tracking.

This paper assesses the psychometric value of allowing test-takers choice in standardized testing. New theoretical results examine the conditions where allowing choice improves score precision. A hierarchical framework is presented for jointly modeling the accuracy of cognitive responses and item choices. The statistical methodology is disseminated in the ‘cIRT’ R package. An ‘answer two, choose one’ (A2C1) test administration design is introduced to avoid challenges associated with nonignorable missing data. Experimental results suggest that the A2C1 design and payout structure encouraged subjects to choose items consistent with their cognitive trait levels. Substantively, the experimental data suggest that item choices yielded comparable information and discrimination ability as cognitive items. Given there are no clear guidelines for writing more or less discriminating items, one practical implication is that choice can serve as a mechanism to improve score precision.

This article reviews recent advances in the psychometric and econometric modeling of eye-movements during decision making. Eye movements offer a unique window on unobserved perceptual, cognitive, and evaluative processes of people who are engaged in decision making tasks. They provide new insights into these processes, which are not easily available otherwise, allow for explanations of fundamental search and choice phenomena, and enable predictions of future decisions. We propose a theoretical framework of the search and choice tasks that people commonly engage in and of the underlying cognitive processes involved in those tasks. We discuss how these processes drive specific eye-movement patterns. Our framework emphasizes the central role of task and strategy switching for complex goal attainment. We place the extant literature within that framework, highlight recent advances in modeling eye-movement behaviors during search and choice, discuss limitations, challenges, and open problems. An agenda for further psychometric modeling of eye movements during decision making concludes the review.

Various recent works have developed “feature” or “aspect” models of similarity and preference. These models are more concerned with the fine detail of the judgment process than were prior models, but nevertheless they have not in general developed an underlying stochastic process compatible with the assumed structure. In this paper, we show that a particular class of multivariate stochastic processes, namely those associated with the Marshall-Olkin multivariate exponential distribution, generates several of these models. In particular, such stochastic processes (appropriately interpreted) yield Tversky's elimination by aspects model, Edgell and Geisler's (normal) additive random aspects model, and Shepard and Arabie's additive cluster model.

Social and naturally occurring choice phenomena are often of the “pick-any” type in which the number of choices made by a subject as well as the set of alternatives from which they are chosen is unconstrained. These data present a special analytical problem because the meaning of non-choice among pick-any choice data is always ambiguous: A non-chosen alternative may be either unacceptable, or acceptable but not considered, or acceptable and considered but not chosen. A model and scaling method for these data are introduced, allowing for this ambiguity of non-choice. Subjects are represented as points whose coordinates are proportional to the centroids of the points representing their choices. Alternatives are represented at points whose coordinates are proportional to the centroids of the points representing subjects who have chosen them. This centroid scaling technique estimates multidimensional joint spaces from the pick-any data.

Any family of simple response time distributions that correspond to different values of stimulation variables can be modeled by a deterministic stimulation-dependent process that terminates when it crosses a randomly preset criterion. The criterion distribution function is stimulation-independent and can be chosen arbitrarily, provided it is continuous and strictly increasing. Any family of N-alternative choice response time distributions can be modeled by N such process-criterion pairs, with response choice and response time being determined by the process that reaches its criterion first. The joint distribution of the N criteria can be chosen arbitrarily, provided it satisfies certain unrestrictive conditions. In particular, the criteria can be chosen to be stochastically independent. This modeling scheme, therefore, is a descriptive theoretical language rather than an empirically falsifiable model. The only role of the criteria in this theoretical language is to numerically calibrate the ordinal-scale axes for the deterministic response processes.

In a ‘very close replication’ study using the same attributes as the original, Chandrashekar et al. (2021) report a failure to replicate some choose–reject problems documented in Shafir (1993). We find that several of the original attributes have changed their valence three decades later, and we compose new versions with updated attributes that fully replicate Shafir’s (1993) original findings. Despite their apparent exactitude, ‘very close replications’ across contexts or time, when stimuli may have changed their meaning or valence, can be highly misleading, further exacerbating replication concerns.

In response to a crisis, policymakers face the decision of whether to enumerate specific actions the public must do or, instead, to aim at an overall outcome while leaving room open for choice. This essay evaluates the merits and demerits of crisis response that leaves room open for choice, with a particular focus on pandemic response. I evaluate two approaches: trades and offsets. Trades allow individuals or groups to exchange protection against harm or entitlement to engage in risky activity. Offsets allow the same actors to pay to mitigate the effects of decisions that increase risk for others. Choice-friendly approaches can free people to better align their actions with their values, harness local knowledge for better social outcomes, and act as natural experiments. However, they also are subject to objections, including negative externalities, agency problems, exploitation, and exacerbating inequality.