8: Ultimate Solution

Opening Data, Materials, and Workflow

Other Information:

The Ultimate Solution: Opening Data, Materials, and Workflow -- Implementing the strategies in the previous section will shift the incentives toward more efficient knowledge accumulation. They do not, however, address the core factor that led Motyl and Nosek to conduct a replication in the opening anecdote—accountability. Science is a distributed, nonhierarchical system. As noted by Nosek and Bar-Anan (2012): "Open communication among scientists makes it possible to accumulate a shared body of knowledge. . . . Individual scientists or groups make claims and provide evidence for those claims. The claims and evidence are shared publicly so that others can evaluate, challenge, adapt, and reuse the methods or ideas for additional investigation. . . . Science makes progress through the open, free exchange of ideas and evidence. (p. 217)"

Stakeholder(s):

• Scientists: Openness provides scientists with confidence in the claims and evidence provided by other scientists. Further, reputation enhancement is a primary mechanism for reward in unstructured contribution systems. Scientists gain and lose status by their public contributions to scientific progress. As such, public reputation management is the primary lever for promoting accountability in academic science.


• Journals: In present research practice, openness occurs almost entirely through a single mechanism—the journal article. Buckheit and Donoho (1995) suggested that "a scientific publication is not the scholarship itself, it is merely advertising of the scholarship" to emphasize how much of the actual research is opaque to readers. For the objective of knowledge accumulation, the benefits of openness are substantial. Openness increases accountability (Lerner & Tetlock, 1999); makes it easier to share, adapt, extend, and critique methods, materials, analysis scripts, and data; can eliminate the file-drawer effect; and can improve the potential for identifying and correcting errors (Ioannidis & Khoury, 2011; Ioannidis & Panagiotou, 2011; Schooler, 2011; Stodden, 2011).

Objective(s):

8.1: Open Data

8.2: Methods & Tools

8.3: Workflow

Content