“We need to move beyond the accept/reject binary in peer review” by George Currie and Damian Pattinson, originally published on LSE Blogs, licensed under CC BY 4.0. No changes made.
Peer review has become an integral part of the current scholarly publishing model. As its suitability as a quality filter comes under increasing scrutiny, George Currie and Damian Pattinson argue it has become wrongly conflated with validation. Considering three common myths sustaining binary peer review, they make the case for an iterative and open review system.
Science is a method of inquiry, not a body of facts. Individual studies offer partial and provisional insights, shaped by methodological and contextual limitations, and scientific understanding develops through replication and ongoing scrutiny.
Yet, decisions about which findings become part of the scholarly record are made through a binary filter: the journal accept/reject decision. This decision, mediated by peer review, can assess whether evidence supports stated conclusions, whether methods are appropriate, and how a study relates to existing knowledge, but it cannot establish whether findings are replicable or represent broader truths.
When the outcomes of this process are interpreted as markers of certainty, challenges arise.
Journals are often perceived as arbiters of trust, and peer review assumed to confer validity that equates to truth. When the outcomes of this process are interpreted as markers of certainty, challenges arise. For research findings communicated beyond academic contexts, this can lead to uncertainty and limitation being downplayed in favour of clearer narratives. This risks overstating the strength or applicability of individual results, while disagreement or later revision is seen as failure, instead of a normal feature of scientific progress.
Building healthier practices in how science is communicated requires greater comfort with nuance and uncertainty, qualities that are not captured in binary validation and publication decisions. At the core of the present system are three myths.
Myth one – Binary decisions are reliable
For publication to be a reliable signal of scientific validity, journal editorial processes and peer review would need to be a consistent filter for unsound research. While individual journals may apply rigorous standards, the scale and diversity of the journal ecosystem mean that, taken as a whole, it functions unevenly and does not provide a reliable system-wide filter. Publication alone is therefore a poor proxy for validity, and passing peer review does not guarantee that a study is scientifically sound.
Retractions have increased in recent years, including cases of serious error and misconduct, demonstrating that flawed or fabricated work can and does pass peer review. Researchers face strong pressures to publish frequently, while at the same time publishers benefit from increased volume of articles. These incentives weaken the effectiveness of journal-level filtering and undermine confidence in binary accept/reject decisions as indicators of scientific reliability.
Myth two – Binary validation is binary
Although journal publication operates through the binary accept/reject decision, in practice, validation is not treated as a binary outcome. Instead, an implicit scale of legitimacy operates through journal prestige, with little attention paid to the intrinsic quality of individual articles. If publication were genuinely binary, all published research would be regarded as equally valid; in reality, journal-level indicators such as impact factor and brand reputation function as proxies for quality, conflating trust in research with trust in particular journals.
This hierarchy is illustrated by journal transfer and cascade models, in which papers rejected from high-prestige titles are passed down through tiers of journals. The result is not a clear distinction between validated and non-validated research, but stratification into tiers of perceived legitimacy. Binary journal decisions therefore serve less as markers of validation than as points of entry into a prestige hierarchy that determines how much weight research is given.
Myth three – Research is either sound or unsound
Binary validation attempts to sort research into categories of valid or invalid, yet reliability exists on a spectrum. Studies differ in statistical power, effect size, and evidential strength, even when methods are sound. A small, well-conducted study and a large-scale study of equal quality may both pass peer review and be labelled “valid”, despite offering very different degrees of confidence. In a binary system, both are signalled as equally sound, and important distinctions in the strength of the evidence they provide are erased.
As well as obscuring nuance, binary thresholds actively favour certain results. Studies reporting positive, confirmatory, or statistically significant findings are more likely to pass peer review and be accepted for publication than those that produce null, negative, or challenging results. This preference, reinforced by opaque editorial decision-making, entrenches publication bias and skews the scholarly record toward apparent consensus. Evidence also suggests that reviewer judgments are often uncertain and probabilistic, but the system forces this nuance into a binary outcome. Ultimately, binary validation offers only a veneer of certainty over a process that is inherently uncertain and evolving.
Open peer review
Taken together, the limitations of binary validation point to the need for evaluation models that better reflect the provisional and cumulative nature of scientific knowledge. One practical starting point is greater use of public or open peer review, which makes assessment visible rather than implicit, and allows judgments to be expressed with nuance and detail. By exposing reasoning, disagreements, and degrees of confidence, open review shifts validation from a single binary outcome to an ongoing process that, importantly, creates distinctions between publication, evaluation, and validation— and making scrutiny, critique, and revision visible parts of the scholarly record.
By exposing reasoning, disagreements, and degrees of confidence, open review shifts validation from a single binary outcome to an ongoing process
While open review does not resolve all challenges associated with research assessment, it provides a foundation for more contextual, and proportionate judgment, addressing some of the biggest limitations of binary validation.
Recognition of peer review regardless of outcomes
When reviews are public, peer review can be recognised and valued even when it does not result in acceptance. In opaque review systems, publication functions as the only visible signal that research has been evaluated. When a paper is rejected, the reviews effectively disappear: while authors may incorporate feedback, subsequent journals have no visibility of prior assessment, and reviewers’ contributions are lost to the record. Each resubmission therefore requires new reviewers to repeat work that has already been done, duplicating effort and obscuring the distinction between research that has not been endorsed and research that has not been evaluated.
Increasing transparency without moving away from notions of binary validation only serves to entrench issues of binary validation and “risks turning peer review reports into promotional endorsements rather than critical evaluations.” Instead, review systems that support portable and traceable evaluation would allow assessment to persist independently of publication decisions. Rather than operating as a pass or fail threshold, review could be recorded as a nuanced mix of critique and endorsement, reflecting both strengths and limitations.
Collaborative and iterative research evaluation
Digital publishing has removed the constraints of print, allowing research to be shared, revised, and re-evaluated as evidence accumulates. Papers no longer need to be fixed at the point of acceptance: authors can respond to critique, update their work, and release new versions that reflect the evolving nature of scientific understanding rather than a single frozen moment.
Binary decisions are inevitable, in the choice to read, cite, or disregard something, but they are not fit as proxies for scientific validity.
Public review complements this flexibility by enabling evaluation to be similarly iterative and distributed. Papers can be assessed multiple times by reviewers with different expertise, supporting interdisciplinary work and allowing reviewers to add value where they are most able without expectations of a final verdict. Making reviews visible preserves reviewer labour, reduces duplication, and provides readers with clearer context for interpreting findings, aligning evaluation more closely with how scientific knowledge is actually produced.
Can we escape binary decisions?
Binary decisions are inevitable, in the choice to read, cite, or disregard something, but they are not fit as proxies for scientific validity. Traditional journals conflate review and curation, making publication the primary signal of quality rather than peer evaluation itself. Public review separates review from validation, preserves the work of reviewers, and allows research to evolve iteratively. Adoption of transparent, iterative, and accountable systems would better reflect the provisional, nuanced, and cumulative nature of science.
