Reproducibility Project

Overview
========
Do normative scientific practices and incentive structures produce a biased body of research evidence? The Reproducibility Project is the first known empirical effort to estimate the reproducibility of a sample of studies from the scientific literature. The project is a large-scale, open collaboration involving dozens of scientists from around the world.

The investigation is currently sampling from the 2008 issues of three prominent psychology journals - *Journal of Personality and Social Psychology*, *Psychological Science*, and *Journal of Experimental Psychology: Learning, Memory, and Cognition*. Individuals or teams of scientists follow a structured protocol for designing and conducting a close, high-powered replication of a key effect from the selected articles. We expect to learn about:

1. The prooverall rate at which peer-reviewed, published psychology studies produce replicable results

1. Obstacles that arise in gathering information about and replicating original study procedures

1. Predictors of replication success, such as the jectournal in will ehich the original finding was published, the citation impact of the original report, and the number of direct or conceptual replications that havaluate the ability to reproduce the original study procedures and the e been published elsewhere

1. Aspects of a procedure that are or are not critical to a successful direct replication, such as the setting, specific characteristics of the sample, or details of the materials.

What our researchers say about the Reproducibility Project
===================

"If [our] discoverall probability of replicating the original resultsies are true, other people in other places should be able to rely on them, and build on them, to push further, and build a cumulative body of knowledge. FurtherThese values are central to what it is, it for me, to be a scientist, and the Reproducibility Project expresses those values."

"This project makes doing replications more rewillarding (you get to work with examine the predictors of replication success - ecellent researchers, you learn a lot about how to do a good replication study, [.g..] if you don't know how to do something, publishingthere is help available). Showing this can be done, and documenting how a replication study should be done, will help other researchers in the future [...] and hopefully, give replications the place in journal, number of conceptual/direct replications in the published literature, citation impact of the original article, closeness of the replication to the original circumstances: sample, setting, materialss they deserve. "

"Interested contributors can still join the project hope that we will inspire researchers everywhere to think more carefully about the decisions that they make throughout the research process and to be more transparent and thoughtful about their research process [...]. I also hope that it will allow people to really appreciate the challenges and complexities of replications and the important value that replications serve for our field."

How to get involved
===================
The Reproducibility Project is most likely to appeal to psychological scientists that are interested in the reproducibility of the field's research results oropen to anyone wanting to participateho is interested in the reproducibility of psychological science or in participating in a large-scale, open science project. Contributors receive authorship on project reports (see [Executive Summary](https://docs.google.com/document/d/1FcWLfASVXPkLuTVQmbZKvpkPsgrW8XKPGfWJqnSnmeM/edit) for details). Ways to contribute include:

The most straightfor - Conducting or helping ward contribution is to conduct one or more replication studies folloith a replication study
- Helping wing the guidelines described in theth statistical analyses or power / sample size calculations
- Researcher Guide. If you are interested as an individual or teamewing and contributing to replication protocols or reports
- Coding published studies (extracting hypotheses, contact the project coordinatorsample characteristics, noseand requirements for replication)
- Programming task at virginia dot edu, with questions or to sign-up and get editing access to the project materials.s such as creating stimulus presentation scripts or automated data processing

There are other If you'd like to get involved, fill out our [neways to contribute including ta contributor survey](https://ucsf.us.qualtrics.com/SE/?SID=SV_1Rl41w596GMLhad) and we'll get in touch with you about opportunities that are a match for your sks appropriate forills and resources. We welcome contributions from students and citizen scientists that do notas well as researchers and scientific professionals. If you have resources for conducting a replicationany questions, feel free to contact our volunteer coordinator at reproducibilityproject@gmail. Some tasks include: coding studies in the eligible pool, creating or improving tables and figures for communicating results, contributing data analytic expertise, and web design for project communicationcom.

Project materials
=================
The Reproducibility Project is open and transparent. You can access background information about the project, materials about the study design, and information about the present progress and results . Our main policy discussions are carried out on an archived, publicly available mailing list. Some key links are belowith the links below. Also, a public google group documents discussion about the project. Anyone can join the google group to monitor the discussion, or contribute.:

1. - [Executive Summary](https://docs.google.com/document/d/1FcWLfASVXPkLuTVQmbZKvpkPsgrW8XKPGfWJqnSnmeM/edit) : Get a- An overview of the project goals and approach (start here if you want some background of the project)'s goals and approach
2. - [Researcher Guide](https://docs.google.com/document/d/1b8wlI8RqR07aOJKv5qMRTGCSi10ntb7glL7gyzUEE3M/edit) : A comprehensi- Steps and tasks inve list of tasks for researchers that would like to conduct a replication (start here if you know the project and want to join in)olved in carrying out a replication study
3. - [Project Progress and Results Spreadsheet](https://docs.google.com/spreadsheet/ccc?key=0AvIo2znxWnxZdERIS2xqNnNxUUZRRTB5LVJxckhiY3c#gid=8) : The primary - Our central listing of avehicle for reailable studies, replications in progress, and contributions by viewing eligible articles, reporting project progress and results, and documenting researcher contributionolunteers
4. - [Google Discussion Group](http://groups.google.com/group/openscienceframework?hl=en) : to participate in the dynamic discussion of pThe discussion list for the Open Science Collaboration (including the Reproducibility Project design and progres). Anyone can join the list or review the archives.
- Project Materials Dropbox: A repository of articles that are eligible for replication and other background reading materials (and discussions about other OSF email reproducibilityproject@gmail.com for access)
5. Project Materials Dropbox: Access scientific articles that are eligible for replication, or other background reading materials (email nosek at virginia dot edu to get access)
6For a comprehensive list of Reproducibility Project resources and documentation, see our [links and resources page](http://openscienceframework. Open Science Frameorg/project/VMRGu/node/xqjeb/worik: Replication study materials will be made available in the OSFi/home).