Recently, I have reviewed quite a few papers, so I decided to write down my thoughts.
When I reviewed a paper for the first time, my focus was on what decision to make—accept, minor/major revision, or reject. However, I realized that this is inherently a very subjective process. I increasingly did not want to miss out on a good paper because of my own mistakes. Over time, after reviewing many manuscripts, I developed my own framework, which is particularly useful for first-time reviewers with little experience.
First, the purpose of reviewing should not be simply to decide on acceptance or rejection. Beginners might find it appealing to judge others from a “god-like” perspective, but I see this as an illusion to avoid. Our starting point when handling a manuscript should be how to best help the authors improve their work. Begin by reading the abstract and introduction—authors will certainly highlight their contributions there. After reading these, you’ll understand what the paper does, its strengths, and what it claims. The next step is to check the literature for similar work. If you find very similar prior work, your review comments must express your concerns or require the authors to add the necessary references. The second part is to evaluate whether the paper’s claims hold up—are they sufficiently convincing? If not, raise questions for the authors. Personally, my reviewing style includes always requiring authors to provide experimental code; I check many aspects, such as whether the experimental section is rigorous. I also go through grammar issues and whether the presentation is clear and smooth. Finally, and most importantly, I believe novelty should be assessed based on the practical value of the work and its future potential. If a paper honestly presents both its strengths and weaknesses, even if its performance is slightly inferior, I will still give it a high score.
It’s also important to note that reviewing is not meant to make things difficult for authors. Some inexperienced reviewers might ask authors to compare with every possible related work. In reality, experiments should focus on the contributions of the paper. Comparing everything can distract from the main point and make the paper excessively long.
As for the final decision—major revision, minor revision, or reject—it should be based on the workload your comments would impose on the authors. Trust that the editor will have a clearer view here than you do.
That’s roughly my experience. I hope it provides some inspiration for readers.