Making data available for replication

Political Analysis requires that authors make replication materials publicly available prior to publication. The data and code for your article must be uploaded to the Political Analysis Dataverse and cited in the final version of your manuscript. Below are instructions on how to do this. We will formally accept and then forward your final manuscript to production only after we have verified your compliance with this requirement.

Authors are encouraged to spend time preparing their replication materials, in particular developing usable documentation for eventual users of their replication package. A replication package should contain:

• A brief “readme” file that summarizes the materials that are part of the replication package.
• Well-documented and well-named code for producing the results reported in the tables and figures of the paper.
• Specialized software packages, modules, or routines that are not a standard component of public-release, off-the-shelf software.
• The data necessary to reproduce the results reported in the paper.
• Documentation so that users know how to use the code and data to reproduce the results reported in the paper.

The replication materials will be released and permanently archived on the journal’s Dataverse. Thus, authors should bear in mind that code, documentation and data will be publicly available, and thus all should be edited carefully. In particular, all data made available in replication packages should be made anonymous, and in general no individually-identifying information should be present in replication datasets. Thus, if your data require confidentiality, you should anonymize the relevant variables or cell values.

Replication materials for all analyses reported in the published version of the paper are subject to this requirement, including (but not limited to) quantitative results, simulations, and qualitative analyses. You are only required to provide enough information to replicate the results in your article, not all the data in your possession or even in your data set. However, the more information you provide, the more likely someone will follow upon your article, which would be good for you, your article, and PA. If you wish to request an exception to this policy, please contact the editors.

Political Analysis Dataverse instructions:

• Go to the Political Analysis Dataverse. Click the "Add Data" button and select “New Dataset” in the dropdown menu.
• Enter cataloguing fields to describe your data file(s), such as title, author name(s), abstract, year, citation to article, etc.
• Scroll down to the “Files” section and click on “Select Files to Add” to upload your data files, code, documentation, and an explanation of what each of the files are. We recommend you upload tabular data files in one of the formats Dataverse presently recognizes, in which case it will process the files and provide additional formats to the end user.
• Click the “Save Dataset” button when you are done. Your unpublished dataset is now created. When the dataset is ready, click "Publish Dataset", then "Send for review" to submit the draft version of the dataset for replication.
• You will receive the citation to your replication data set when you upload it. Please insert the complete citation in your manuscript’s references, and refer to this in both the “Data Availability Statement” section within your manuscript and in a footnote in your manuscript around where you first describe your data or analysis.
• When you have completed this process please email confirmation to politicalanalysis@cambridge.org.

After uploading the replication materials, they will be reviewed for completeness and eventually released for public use on the Political Analysis Dataverse page. Authors are encouraged to return to their study’s Dataverse entry after their paper has been published, and to update the Dataverse entry.

Editorial notes on replication results

We will attempt to use the replication materials you have deposited in the Political Analysis Dataverse to replicate the results reported in your manuscript. In some cases we may not be able to complete this check for analyses involving large datasets, proprietary APIs, or restricted-access data. In such cases, a successful replication of a subset of the results or data may be sufficient. The scope of the replication check may range from a full reproduction of all results to checks on a subset of the data or code.

The editorial team will add a brief note about the results of the replication check to each published article, summarizing its scope and outcomes. These may include exact replication, replication to within approximation error, minor discrepancies without substantive impact, or, in exceptional cases, a failure to reproduce the reported results.

This Replication Note from the Editors will be in he format, "The Political Analysis replication team [outcome clause], [scope clause]." During the submission process, you will be asked to confirm that you consent to a Replication Note from the Editors being added to your article which may include any of the following clauses:

Outcome clauses

• "successfully replicated all results exactly": all numerical values and figures match those reported in the manuscript.
• "successfully replicated all results to within approximation error": minor numerical differences are present. These are typically due to random seeds, hardware, or other stochastic processes.
• "replicated most results; some discrepancies were observed, but they are not substantively meaningful": differences exist but do not affect the substantive conclusions of the manuscript.
• "was unable to reproduce the results reported in the manuscript": results differ in substantively meaningful ways. This outcome is rare, and manuscripts may be rejected or returned for further review rather than published with this clause.

Scope clauses

• "using the full replication archive": all provided materials were executed as submitted.
• "using a subset of the data provided by the authors": replication was limited to a partial dataset due to size, computational, or other constraints.
• "using only the publicly available components of the archive": restricted-access materials were not replicated; some of the data used in this research could not be obtained by the replication team.
• "using restricted-access data accessed in a secure environment": replication was conducted in a controlled setting by the replication team, but some data and/or code cannot be included in the public replication archive.
• "using portions of the analysis due to reliance on third-party APIs": replication was limited to analyses not dependent on API calls, for example due to the cost or instability of third-party APIs.

Research preregistration

Political Analysis encourages authors to consider preregistering their studies, when appropriate. Preregistration is the act of archiving a research design with a third party prior to observing a project’s outcome variables. Releasing precise information about hypotheses, how they will be tested, and any pre-outcome data all serve to raise the level of transparency in the project. The goal of preregistering a study is to communicate research goals and strategies as clearly as possible before the outcome variable is observed, allowing readers to distinguish between analyses specified ex ante from those crafted as a function of outcomes.

Research designs should be deposited prior to analysis with a registry that: is open to all prospective registrants; requires that at minimum researchers provide a description of the intended research, a description of hypotheses or other conclusions that the research seeks to examine, a description of data sources including, as applicable, site, subjects, and timeframe, a description of the methods to be used, a description of whether outcomes have been realized prior to registration, and contact information for a lead researcher; records the date and time of all registered research designs and subsequent modifications of designs; provides all registered designs with a unique identifier; makes metadata publicly and freely accessible; and can provide journals with access to complete data at the time of article submission, and to the public within at least two years of completion of data collection.

At this point in time, we encourage the use of the following repositories for preregistered study designs:

• The Political Science Registered Studies Dataverse
• The American Economic Association’s RCT Registry
• The Registry for International Development Impact Evaluations (RIDIE)

A link to the preregistered study should be provided to the editors upon submission of the paper. The author should indicate if they would like reviewers to be able to review the preregistered information, and whether or not the preregistered information has been made anonymous. Preregistered studies also should include a link to the preregistered information in the final published article, in the same footnote as the link to the registration data. Authors should discuss in detail any deviations from the registered design, their rationale for those deviations, and the implications of these deviations on the reported results.