Stage 5: Review Publishing
Publish peer review Other Information:
Publishing Peer Review -- Peer review is a central scientific practice (Peters & Ceci, 1982). Confidence in the quality of
design, analysis, and interpretation is improved by the independent evaluation of expert peers. Whereas ultimate confidence
is in the reproducibility of the results, and by the eventual impact of the research and theory on other science, peer review
presently serves as a quality control barrier to entry (Armstrong, 1997; Goldbeck-Wood, 1999; Horrobin, 1990). An editor and
one to five reviewers decide whether the scientific community should see, read, and be influenced by the work—at least in
their journal.
Stakeholder(s):
- Journal Editors
- Journal Reviewers
- Peer Reviewers: In addition to the gatekeeping (removed in Stage 3) and evaluation (changed to a grading system in Stage 4), excellent peer
review can identify confounds or problems in the research design, point out alternative analysis strategies that avoid inferential
challenges, suggest new avenues of investigation that can clarify the validity or applicability of the hypothesis, or provide
alternative theoretical understandings of the same empirical evidence. In other words, peer review can contribute to scientific
progress. However, present practices do not take full advantage or give recognition to these contributions of peer review.
Stage 5 corrects this by publishing peer reviews (see also Benos et al., 2006; Wicherts, Kievit, Bakker, & Borsboom, 2012).
Reviewers conduct reviews for review services. Reviewers then decide if their reviews will be published in the repository
alongside the originating article. If authors revise and resubmit the article, new reviews are attached to the resubmitted
version. Readers have access to the evolution of the article and the reviews from each stage. In present practice, there are
few incentives to be a reviewer and to do a good job reviewing. Peer review is voluntary and usually anonymous. Peer review
takes time. The most that it can do for reputation building is add a minor vita entry. Further, when reviewers do invest time
into the process and provide an excellent, insightful review, the only knowledge gain from that effort is for the authors,
editor, and other reviewers. Only a portion of that scholarship influences the manuscript, and the reviewer gets no identifiable
credit for the contribution. As frequent reviewers, we have observed reviews by others that provided us with insight and ideas
that would surely have benefited others. Besides being a loss of scientific contribution and a disincentive for doing a good
job, the closed nature of the peer review process violates the scientific values of openness and transparency.
- Scientists: Stage 5 increases the incentives for high-quality peer review. Having given up lucrative careers doing something else, the
scientist's primary currency is reputation. Scientists build reputation by contributing to science, primarily through publication.
Stage 5 creates a new category of scientific contribution by giving peer reviewers the opportunity to publish their reviews.
If reviewers choose to do so, the reviews are published under their name and are linked to the version of the report that
they reviewed. The original authors have the opportunity to address those concerns whether they publish a revision or not.
Reviews become a public scientific contribution, and scientists can gain reputation by being good reviewers. This already
occurs in mathematics. The majority of math articles have published reviews that appear in Mathematical Reviews (http://www.ams.org/mr-database/)
that identify the reviewer and are a basis of reputation building. Likewise, the journal Biology Direct (http://www.biology-direct.com/)
publishes reviews and the authors' responses to the reviews (e.g., http://www.biology-direct.com/content/7/1/4).
- Research Contributors: Publishing reviews would increase transparency and would make anonymous reviews infrequent rather than the norm. It would
also be much easier to learn about the strengths and limitations of articles. Reviews often raise interesting limitations
and open questions but recommend publication nonetheless. Because the critical points are public, authors would care more
about addressing critical points rather than focusing on doing just enough to convince the editor. Finally, a new class of
contributor would emerge—scientists who rarely do their own novel research but frequently offer critique and review of others.
These contributors already exist, but they are unrecognized because the present system does not acknowledge or reward their
contribution. This is underused potential, particularly considering that most trained scientists are not at high-output research
universities generating research. But they are trained, knowledgeable, and can offer great insight on what is produced by
others. Further, it is likely that high-quality critiques would be cited by later articles either to raise or address the
critique.
- Research Evaluators: It is easy to conceive of a future in which some scientists could earn tenure by being renowned evaluators of research rather
than producing research on their own.
Objective(s):
|
