6: Peer Review Standards

[Develop] peer review standards focused on the soundness, not importance, of research

Other Information:

Journals with peer review standards focused on the soundness, not importance, of research -- The basis of rejection for much research is that it does not meet the criterion of being sufficiently "important" for the journal considering it. Many manuscripts are rejected on this criterion, even if the reviewers identify the research as sound and reported effectively. Despite evidence of the unreliability of the review process for evaluation and identifying importance (Bornmann, Mutz, & Daniel, 2010; Cicchetti, 1991; Gottfredson, 1978; Marsh & Ball, 1989; Marsh, Jayasinghe, & Bond, 2008; Peters & Ceci, 1982; Petty, Fleming, & Fabrigar, 1999; Whitehurst, 1984), this is a reasonable criterion given that journals have limited space and desires to be prestigious outlets. However, in the digital age, page limits are an anachronism (Nosek & Bar-Anan, 2012). Digital journal PLoS ONE (http://plosone.org/) publishes research from any area of scientific inquiry.

Stakeholder(s):

• Journals


• Peer Reviewers


• PLoS ONE: Peer review at PLoS ONE is explicitly an evaluation of research soundness and not its perceived importance. Since its introduction in 2006, PLoS ONE's growth has been astronomical. In 2011, 13,798 articles were published (70% acceptance rate), making it the largest journal in the world. Given the disregard for importance in the review process, one might surmise that PLoS ONE's impact factor would be quite low. In fact, its 2011 impact factor was an impressive 4.41. This put it in the top 25% of general biological science journals and nearly as high as Psychological Science (4.7). This casts further doubt on reviewers' ability to predict importance (Gottfredson, 1978) or at least one indicator of importance: citation impact. With a publishing model focused on soundness, negative results and replications are more publishable, and the journal identity is not defined as publishing research that is otherwise unpublishable.


• Psychological Science


• Biological Science Journals

Objective(s):

1

Content