The Editorial Staff of the American Journal of Political Science is pleased to announce important new revisions to the policy regarding replication materials for AJPS articles. These innovations represent significant changes to current operations. Research transparency and replicability of results are standards to which the discipline traditionally has paid lip service. The new AJPS replication policy requires scholars to “practice what we preach” and adhere to these standards in a meaningful way
For many years, authors of manuscripts accepted for publication in the AJPS were asked to make their data available to interested researchers for replication purposes. Authors were strongly encouraged to comply with this policy, but doing so was voluntary on their part. As a result, the availability and quality of replication materials varied enormously across articles.
Former AJPS Editor Rick Wilson established a new policy by requiring authors to upload replication files to the American Journal of Political Science Dataverse on the Harvard Dataverse Network (https://thedata.harvard.edu/dvn/dv/ajps). This important innovation guaranteed that materials would be available for all AJPS articles (or at least those using empirical data) in a single, easily accessible location. To our knowledge, the American Journal of Political Science was the first publication outlet in the discipline to implement such a requirement for authors.
While the creation of the AJPS Dataverse was a critical first step, some problems remained that limited its effectiveness. Specifically, authors were given very little guidance regarding exactly what information and materials they needed to provide. As a consequence, the unfortunate variance in the content of replication files across articles persists.
Currently, we are addressing the previous issue through two important and related new innovations. First, a new document– “The American Journal of Political Science Guidelines for Preparing Replication Files” (CLICK HERE to download)– informs authors about exactly what information they are expected to provide and upload to the AJPS Dataverse. The Guidelines explain that replication materials must contain: The analysis dataset; detailed, clear code for using the analysis dataset to reproduce all tables, figures, and exhibits in the AJPS article; information about the source of the analysis dataset; and any instructions for extracting the analysis dataset from the source data (e.g., recodes, data transformations, details about missing observations, etc.).
We understand that these guidelines probably will not cover every possible situation that may arise. But, they should be sufficient for most data analyses that are reported in AJPS articles. The objective is to establish a general standard for the information that must be made available to the research community showing how to replicate the work that appears in the pages of the American Journal of Political Science.
The second major innovation is that submitted replication materials will be verified to guarantee that they do, in fact, properly reproduce the analysis results. The Midwest Political Science Association and the Editorial Staff of the American Journal of Political Science have entered into an arrangement with the University of North Carolina’s Odum Institute for Research in Social Science to carry out the verifications as a regular component of the publication workflow. When the AJPS Editor tentatively accepts a manuscript for publication, the author is informed that final acceptance is contingent upon successful replication of the results. The author is instructed to upload replication files to the AJPS Dataverse at the same time that the final draft of the manuscript is submitted.
When the author provides the final draft, the AJPS Editor notifies the Odum Institute staff about the new study. They verify that the full set of replication files has been provided, and that the files can be used to reproduce all data analyses and results from the article. If problems occur, then Odum staff inform the AJPS Editor, who gives the manuscript author the opportunity to resolve the issue and upload corrected materials to the Dataverse. This process continues until the replication is carried out successfully. At that time, the AJPS Editorial Staff sends the final version of the article to the publisher for copy editing and eventual publication, first online and then in a print issue of the AJPS.
We believe that this new AJPS policy makes an important contribution to the infrastructure of social scientific research. The procedures guarantee that complete replication materials are available and operate satisfactorily for all articles that appear within the Journal‘s pages. Moreover, given the American Journal of Political Science‘s position as one of the most prestigious, general-audience publication outlets in the discipline, the new replication policy facilitates transparency in, and helps to improve the integrity of, political science research more generally. Thus, the AJPS replication standards should contribute directly to a rigorous, rational, theory-driven, and cumulative approach to understanding political and social phenomena.
Any questions about the replication policy may be directed to the AJPS Editorial Staff at firstname.lastname@example.org.