Who is Reviewer 2? Some truths about peer review
For those innocent lambs out there, Reviewer 2 is the nickname that academia has given to the peer reviewer of your manuscript who gives a laundry list of criticisms, demands eight months of new experiments totaling $300,000 in supplies alone, insults your knowledge of the literature or ability to spell your name correctly, insists your study is boring, or some combination of the above. They lurk at the core of the peer review process.
Terrible people, right? But also a reminder of what we should expect from ourselves when we do peer review.
As a handling editor for major scientific journals over the years, I’ve learned a few things about Reviewer 2. One of the main things I’ve learned is that they’re disproportionately often an early career scientist, such as a postdoc or grad student.
Most of the time, the people who review for me are great, but sometimes I've wondered if my job as editor is to stop cycles of pettiness (generational trauma?) in peer review. It can clearly take hold early. And I get it: I've had some mean things said not just about my work but me in anonymized review. It can feel unfair when reviewers hold our work to what seem like extremely high standards, and we can't then apply the standards to others—or when we see other papers getting published despite flaws that we’ve felt punished for.
We absolutely should insist on rigorous science: Good questions, good design, appropriate statistics, and careful inferences and framing. I worry every day about what I see published.
But that doesn't mean we should criticize a study for being imperfect or not what we hoped it would be. The purpose of a review is not to point out all the flaws in a study but to identify and contextualize its contributions. Put differently, a study can have significant limitations and still be worth publishing. A few small errors are not proof of negligence and shamefulness of the authors. If the design or statistics are so weak that the contributions are negligible, then of course it’s important to say so. But most papers (that I hope you’re reading, anyway) aren’t that bad.
Some other, related reviewer phenotypes I see:
Insecure reviewers: Often they try to hide what they don't understand. This can show up as bizarre criticisms. PSA: If you can't assess some aspects of the methods (certain assays, statistics, etc.), let the editor know in a separate comment which parts of the study you feel capable of critiquing, and don't try to critique the other parts. Editors often invite reviewers that we know have narrow expertise, relative to the study, because their opinions are still valuable.
Arrogant and/or lazy reviewers: They say something is unoriginal, well known, clearly wrong, etc., and then don't cite anything. They state emphatically that a manuscript should be accepted or rejected in the body of their review. Often they don't explain their reasoning or provide any indication they deeply understood the paper.
You-must-be-a-really-unhappy-person reviewers: They make ad hominem attacks. Or they promise me a review several times and then blow off deadlines and ghost me while I deal with understandable frustration from the authors. It’s one of the worst parts of being an editor.
How can we avoid becoming Reviewer 2? When I review, I try my hardest to be compassionate to the editor and authors. I try to explain to the editor what the authors did: What exactly can be gleaned from their study, whether it’s important, and whether the conclusions are sound. I back up my claims with citations. I try to let the authors and editor know what improvements would make the study convincing if it is not. I decline review opportunities if I think I can’t be impartial due to the authors’ identities, if I’m feeling pressed for time (which makes me cranky and raises my standards), or if the study makes me nervous about getting scooped.
As an editor, I try hard not to let inappropriate reviews get sent out and to hold reviewers accountable. Often these reviewers are just starting out and might be accustomed to tear-everything-apart discussions in journal clubs. The existence of the Reviewer 2 trope is a good reminder that we can be kind while be rigorous, and we should be open to new evidence of varying strengths, potentially presented differently from how we would do it.
Fair peer-review is one of the best contributions you can make as a scientist.