The increasing multimodality (e.g., images, videos, links) of social media data presents opportunities and challenges. But text-as-data methods continue to dominate as modes of classification, as multimodal social media data are costly to collect and label. Researchers who face a budget constraint may need to make informed decisions regarding whether to collect and label only the textual content of social media data or their full multimodal content. In this article, we develop five measures and an experimental framework to assist with these decisions. We propose five performance metrics to measure the costs and benefits of multimodal labeling: average time per post, average time per valid response, valid response rate, intercoder agreement, and classifier’s predictive power. To estimate these measures, we introduce an experimental framework to evaluate coders’ performance under text-only and multimodal labeling conditions. We illustrate the method with a tweet labeling experiment.

Many studies argue that the World Bank grants favorable loan conditions to allies of its powerful principals. These studies typically use the count of conditions as a proxy for how demanding loans are on borrowers, even though some conditions are more difficult to comply with than others. We propose a new operationalization: a measure of conditionality stringency in Bank loans constructed using Latent Semantic Scaling. Using this new measure, we find little evidence of a generalizable influence of powerful principals. Instead, the stringency of loan conditions is associated with bureaucratic assessments of risk. To facilitate future research, we provide a new dataset of World Bank loan condition texts and our measure of text stringency for all loans in the dataset.

How do international crises unfold? We conceptualize international relations as a strategic chess game between adversaries and develop a systematic way to measure pieces, moves, and gambits accurately and consistently over a hundred years of history. We introduce a new ontology and dataset of international events called ICBe based on a very high-quality corpus of narratives from the International Crisis Behavior (ICB) Project. We demonstrate that ICBe has higher coverage, recall, and precision than existing state of the art datasets and conduct two detailed case studies of the Cuban Missile Crisis (1962) and the Crimea-Donbas Crisis (2014). We further introduce two new event visualizations (event iconography and crisis maps), an automated benchmark for measuring event recall using natural language processing (synthetic narratives), and an ontology reconstruction task for objectively measuring event precision. We make the data, supplementary appendix, replication material, and visualizations of every historical episode available at a companion website crisisevents.org.

The influence of congressional primary elections on candidate positioning remains disputed and poorly understood. We test whether candidates communicate artificially “extreme” positions during the nomination, as revealed by moderation following a primary defeat. We apply a scaling method based on candidates language on Twitter to estimate positions of 988 candidates in contested US House of Representatives primaries in 2020 over time, demonstrating validity against NOMINATE (r > 0.93) where possible. Losing Democratic candidates moderated significantly after their primary defeat, indicating strategic position-taking for perceived electoral benefit, where the nomination contest induced artificially “extreme” communication. We find no such effect among Republicans. These findings have implications for candidate strategy in two-stage elections and provide further evidence of elite partisan asymmetry.

Previous research finds that policy complexity affects important political processes including legislative delegation and policy diffusion. However, policy complexity is not directly observable and the search for a reasonable proxy constitutes a major challenge for scholars. This research note presents a concise and measurable definition of complex policy based on two aspects: a policy's textual sophistication and its ties to other rules and regulations. Using crowdsourcing and a pairwise comparison framework it is shown that the proposed defining features are crucial for humans’ understanding of policy text. The proposed definition is then operationalized using a large corpus of European Union rules and is shown to outperform alternative operationalizations of policy complexity in predicting the level of legislative delegation.

Research on Russian troll activity during the 2016 US presidential campaign largely focused on divisive partisan messaging. Here, we document the use of apolitical content—content that could counteract mobilization efforts and escape detection in future campaigns. We argue this resembled techniques used by autocratic regimes domestically, in “flooding” social media with entertainment content to distract from and displace mobilizing messaging. Using automated text analysis and hand coding to construct a timeline of IRA messaging on Twitter, we find left-leaning trolls posted large volumes of entertainment content in their artificial liberal community and shifted away from political content late in the campaign. Simultaneously, conservative trolls were targeting their community with increases in political content. This suggests the use of apolitical content might be an overlooked strategy to selectively manipulate levels of attention to politics.

Most authoritarian countries censor the press. As a response, many opposition and independent news outlets have found refuge on the Internet. Despite the global character of the Internet, news outlets are vulnerable to censorship in cyberspace. This study investigates Denial-of-Service (DoS) attacks on news websites in Venezuela and details how news reporting is related to DoS attacks in an attempt to censor content. For this empirical test, I monitored 19 Venezuelan news websites from November 2017 until June 2018 and continuously retrieved their content and status codes to infer DoS attacks. Statistical analyses show that news content correlates to DoS attacks. In the Venezuelan context, these news topics appear to be not only on protest and repression but also on opposition actors or other topics that question the legitimacy of the regime. By establishing these relationships, this study deepens our understanding of how modern technologies are used as censorship tools.

Although attention to populism is ever-increasing, the concept remains contested. This paper provides a comprehensive overview of populism research and identifies tendencies to a conflation of host ideologies and populism in political science through a two-step analysis. First, we conduct a quantitative review of 884 abstracts from 2004 to 2018 using text-as-data methods. We show that scholars sit at “separate tables,” divided by geographical foci, methods, and host ideologies. Next, our qualitative analysis of 50 articles finds a common conflation of populism with other ideologies, resulting in the analytical neglect of the former. We, therefore, urge researchers to properly distinguish populism from “what it travels with” and engage more strongly with the dynamic interlinkages between thin and thick ideologies.

Twitter is a prominent communication tool for politicians with two potential uses: as a “substitute” channel to circumvent constraints from other political arenas, or as an “amplifier” that reinforces party messages. Using a novel dataset containing tweets and parliamentary speeches by members of parliament (MPs) in seven countries, we estimate politicians’ positions and intra-party dissent on European integration. We find that MPs’ sentiment about Europe on Twitter is a valid measure of their party's position, while also uncovering intra-party disagreements. Our results suggest that most MPs amplify the partisan message, but MPs who participate less in parliamentary debate tend to have larger differences with their party on Twitter. Social media thus can free politicians from their party's grip.

Electoral violence is conceived of as violence that occurs contemporaneously with elections, and as violence that would not have occurred in the absence of an election. While measuring the temporal aspect of this phenomenon is straightforward, measuring whether occurrences of violence are truly related to elections is more difficult. Using machine learning, we measure electoral violence across three elections using disaggregated reporting in social media. We demonstrate that our methodology is more than 30 percent more accurate in measuring electoral violence than previously utilized models. Additionally, we show that our measures of electoral violence conform to theoretical expectations of this conflict more so than those that exist in event datasets commonly utilized to measure electoral violence including ACLED, ICEWS, and SCAD. Finally, we demonstrate the validity of our data by developing a qualitative coding ontology.

Contemporary dictionary-based approaches to sentiment analysis exhibit serious validity problems when applied to specialized vocabularies, but human-coded dictionaries for such applications are often labor-intensive and inefficient to develop. We demonstrate the validity of “minimally-supervised” approaches for the creation of a sentiment dictionary from a corpus of text drawn from a specialized vocabulary. We demonstrate the validity of this approach in estimating sentiment from texts in a large-scale benchmarking dataset recently introduced in computational linguistics, and demonstrate the improvements in accuracy of our approach over well-known standard (nonspecialized) sentiment dictionaries. Finally, we show the usefulness of our approach in an application to the specialized language used in US federal appellate court decisions.

Event data provide high-resolution and high-volume information about political events and have supported a variety of research efforts across fields within and beyond political science. While these datasets are machine coded from vast amounts of raw text input, the necessary dictionaries require substantial prior knowledge and human effort to produce and update, effectively limiting the application of automated event-coding solutions to those domains for which dictionaries already exist. I introduce a novel method for generating dictionaries appropriate for event coding given only a small sample dictionary. This technique leverages recent advances in natural language processing and machine learning to reduce the prior knowledge and researcher-hours required to go from defining a new domain-of-interest to producing structured event data that describe that domain. I evaluate the method with the production of a novel event dataset on cybersecurity incidents.

Analyzing audiovisual communication is challenging because its content is highly symbolic and less rule-governed than verbal material. But audiovisual messages are important to understand: they amplify, enrich, and complicate the meaning of textual information. We describe a fully-reproducible approach to analyzing video content using minimally—but systematically—trained online workers. By aggregating the work of multiple coders, we achieve reliability, validity, and costs that equal those of traditional, intensively trained research assistants, with much greater speed, transparency, and replicability. We argue that measurement strategies relying on the “wisdom of the crowd” provide unique advantages for researchers analyzing complex and intricate audiovisual political content.