How to peer review a scientific manuscript
Peer review is one of the most important and, in my view, noble activities we engage in as scientists. How we peer review has a profound effect on the scientific culture of our field. It’s a good skill to master.
My credentials: I’ve served as editor for several high-impact journals with broad scientific coverage and ad hoc editor for several more. As a full professor, I’ve engaged in peer review countless times as an author and reviewer, and I’ve trained people in how to do it.
How can I start peer reviewing?
One way to be invited to peer review is to publish yourself. This isn’t foolproof, but it can get you on authors’ radar. They might cite your paper in theirs, and they might then suggest your name to the editor as a potential reviewer (or “referee”) when they submit their manuscript to the journal.
You might also get on their radar if you wax enthusiastically about their results or ask particularly flattering questions after their conference talks. I’ve also seen this happen from social media posts too: If you repost someone’s Bluesky or LinkedIn post with “Amazing study! Most important of the year!” they can’t help but fantasize you’re available to relay such enthusiasm to an editor. Of course, there’s a special place you-know-where for people who might fake enthusiasm in an attempt to pad their CV with review opportunities.
Probably the easiest and fastest way to start peer reviewing is to ask your advisor for opportunities. Most PIs I know receive at least one request per week, which we mostly decline. We are often racking our brains to suggest alternate reviewers, and sometimes we unfortunately forget the people under our noses or assume they are uninterested.
If your advisor says they are reluctant to recommend you for peer review, ask them why. The most common reason I see PIs discourage lab members from peer review is because they’re not making sufficient progress on their research. It’s better to have a frank discussion about this. If you’re a postdoc and your PI is perennially requesting you not spend time on peer review, I would be concerned about their mentoring intentions.
I accepted a request to review an article, now what?
If it's your first time, I would ask a mentor (ideally one with experience as an editor) to help you write the review. In practice this can mean they read the manuscript and discuss your draft review with you. Journals are fine with coauthored reviews, but you always have to ask for permission in advance. The only reason they might decline is in the off chance that your advisor has a conflict of interest (“COI”), e.g., if they’ve written a paper or shared a grant with one of the study authors in the past few years. In such situations, they might be able to suggest someone else who can help.
If you’re new, I would plan a full day or two (or hourly equivalent) to review the manuscript. I would block off this time in your calendar. If you realize you can’t make the journal’s deadline, it’s better to ask now for an extension. Most editors are so happy to grant such extensions early on, and some journals have now allowed them to be requested via an automated link. Last-minute requests or (worse) late reviews can create memorably unpleasant situations for editors.
What do I put in the review?
The main purpose of a review is to contextualize the paper's contributions for the editor. Again, you're not writing your review for the authors; you're writing for the editor. This flies in the face of common intuition for many, but it can actually lead to much more helpful and respectful reviews. You are not being asked to render a verdict on the paper but to highlight what it offers more and less well.
Specifically,
Demonstrate to the editor in the first paragraph or two that you understand the paper. Give extra context for its significance (or lack thereof) in light of what else is known in the field.
Then list the major strengths and limitations of the paper. If the limitations are minor and the work is solid, say so! You won’t look stupid for liking a manuscript and not finding major flaws in it. (Remember, junior researchers often bring a bit more Reviewer 2 energy than needed.)
Be polite and professional. If you say that something has been done before, cite the papers that did it before.
The point is not to list all the problems and flaws in the paper.
Resist the urge to recommend the authors write the paper you would have written. Evaluate the paper on its own merits. Is it solid scientifically?
You do not need to copy edit. You can put minor corrections and suggestions in a "minor" section at the bottom, if you want.
Do not recommend acceptance or rejection in the review itself. This is a decision for the editor.
In technical fields especially, it's also good practice to let the editor know which parts of the paper you don't feel expert reviewing (e.g., certain lab methods, stats, etc.). There’s often a special box in which to add this information. There’s zero shame in highlighting where your knowledge is shaky.
Do not under any circumstance use a LLM for more than light editing/proofreading. More below.
Time-saving tip: More journals are moving toward structured templates, so check if one exists before writing a free-form review.
Pro tip: If you’re reviewing a manuscript for a journal that does not often publish manuscripts like this, or if you can tell that the editor is from another field, pay particular attention to describing what other work has been done in the field and why this kind of work is important. This is loosely called “educating the editor,” and it can be an important way for your subfield to gain recognition. Journals are not remotely evenly balanced in their expertise and interests, even within their professed scope.
Do I actually have to check all the code/re-do all the derivations?
No. But it’s good if you look through it, and many reviewers check that code is documented and can run as expected.
Also check that the code and other methodological details are available, which ensures reproducibility. Some journals don’t require this on first submission, but I think it’s reasonable to pressure authors and journals to make them available. I never review for journals that do not require data and code be made available on publication. “Made available by authors on request” promises no longer cut it either.
I hope needless to write, you do have to read everything. The supplement and methods are not optional. I also check that key citations are included. It’s not reasonable to “fact-check” all citations, however. If something seems off (“I don’t remember that paper making that claim”), look into it, if you can. As a reviewer, you are not implicitly vouching that the entire paper is completely correct, but you are giving the editor your honest assessment based on your current expertise.
I’ve heard reviews are graded. Is that true? Do I need to impress the editor?
Many editorial systems allow editors to “grade” reviewers on a simple scale using a drop-down menu. In practice, many editors don’t bother. These systems also track the number of times different people have accepted or declined invitations and the number of days they are early or late with their review.
Editors do remember who writes really good and really bad reviews. I can remember years later particularly perceptive and thoughtful reviewers. I am also hearing horror stories from colleagues now about reviews that appear to be partly or entirely written by LLMs. I cannot emphasize enough how using LLMs this way can damage your professional reputation forever.
Cross-review also increase your exposure, good or bad, as a reviewer. Cross-review is when the reviewers have an opportunity to comment on each others’ reviews, usually with the aim of working out issues amongst themselves so the editor can write a streamlined decision letter. In some journals, such as eLife, reviewers know each others’ identities. It has been illuminating sometimes to see how carefully or recklessly different colleagues treat peer review. For the most part, however, I like cross-review because it promotes collegiality, accountability, and clarity in feedback to the editor and authors.
I often use peer review to screen potential postdoc candidates for my lab. Sometimes I’ve received a manuscript that is in the wheelhouse of someone I’ve contemplated recruiting to my lab (or occasionally to the department as an assistant professor). Reviews obviously provide insight into how people think scientifically, and they can also reveal reviewers’ temperaments as colleagues.
Does it matter if I review for MDPI, Frontiers, Elsevier, Springer Nature, Wiley, …?
Yes.
Reviewing for predatory or marginally predatory journals provides free service to journals that don’t help the scientific enterprise, and it doesn’t look good on your CV. MDPI and Frontiers journals are widely considered predatory or at least very low quality. Your efforts best serve the scientific community elsewhere. If you’re not sure if a journal is predatory, look it up and ask around.
Reviewing for Nature, I admit, does often look good on CVs. It looks good out of proportion to the actual merit involved, to be frank. But I would think extremely hard before reviewing for a for-profit publisher. I don’t have room to build a thorough case here, but these publishers (which often have ~40% profit margins) charge extortionate prices to university libraries, to authors, and effectively to taxpayers. They increase the cost of research and ultimate exclude much of the global research community from access to articles. Beyond their access and publication fees, some of them perpetuate additional inequities in publishing (some Nature editors routinely contact certain PIs to see what manuscripts they might have coming—an elitist practice, to say the least). I have great respect for trainees and scientists who allocate their review efforts and submit their best manuscripts to society and non-profit journals. Think PLOS, PNAS, eLife, etc.
I think what another reviewer wrote is wrong. What do I do?
When an editor makes a decision, they usually copy the reviewers on the decision letter, which contains the reviews.
As a reviewer, I have occasionally seen other reviews that revealed deep confusion about the paper or the field. In those cases, I emailed the editor with my concerns about the other review and presented a careful argument. If you care about good science, I think it’s important to do this. You’d want someone to do the same for your paper! This also helps “educate” the editor about the field, or at least which reviewer should not be invited to review this kind of manuscript again.
Why are all the figures at the end of the manuscript instead of inline with the text? It’s so annoying to review.
This is a terrible practice enforced by some journals. You can help. I know some professors have taken to returning manuscripts formatted this way that they have agreed to review. At the very least, you can leave a note for the editor. Usually it’s inappropriate to blame the authors.