Peering into Peer Review: How can you participate in peer review?

As part of the ALiEM Faculty Incubator Program, Dr. Anthony Artino, Deputy Editor of the Journal of Graduate Medical Education (JGME) and Assistant Editor for Academic Medicine participated in a Google Hangout with Drs. Antonia Quinn and Teresa Chan where he provided expert insights into the peer review process. His advice and best practices are summarized below. In this interview, the discussion provides an insider perspective to the peer review process and peer reviewer development. This post is the first half of a longer interview with Dr. Artino with the second half discussing addressing peer review comments.

Outline of the Google Hangout Discussion

• 02:40 What is the purpose of peer review from the journal’s perspective?
• 04:13 What does the initial staff triage look for in a manuscript?
• 05:01 After the staff triage, what does the editor look for in a manuscript?
• 06:36 Knowing the initial manuscript review, what should authors consider prior to submission?
• 07:49 What advantage do you have as a peer reviewer?
• 11:00 What training is provided to peer reviewers?
• 12:00 How can mentors develop new peer reviewers?
• 12:49 What is group peer review?
• 14:06 How can group peer review be initiated?
• 15:30 What do authors need to know about group peer review?
• 15:46 How can mentors advocate for mentees who participate in group peer review?
• 17:07 What resources are available to organize peer review?
• 19:37 Is local peer review recommended prior to journal submission?
• 21:20 What are some perspectives about blinding in peer review?
• 23:50 What are the benefits of peer reviewers signing reviews?
• 25:11 What should junior peer reviewers consider about signing reviews?

Purpose of Peer Review

The purpose is to identify high quality research on current topics and to provide feedback. 

• Will this manuscript advance the literature? Journals want to publish articles to advance the literature. Topics should not only be important and relevant to recent discussions, but also impact the reader’s views of an issue. Is this paper going to change how we think about this area?
• Is the research design and methodology any good? Usually, journals are looking for rigorous and appropriate design, methodology, and argument. Fatal flaws need to be identified in the study or the argument the authors make.
• How can this manuscript be improved? The peer review process is aimed to provide authors feedback for improvement, regardless of journal acceptance.

Peer Review Steps

The following selection process is based on the procedures at the Journal of Graduate Medical Education (JGME). Many of these processes are generalizable to other journals.

1. Staff Triage

Professional staff editors trained in writing and communication initially review manuscripts. Submissions are rejected if they do not fit specific criteria. The following are some examples:

• Is the subject appropriate for the journal? For JGME, is the paper about graduate medical education (GME)? If it is an undergraduate medical education (UME) paper, then the paper will be rejected. There are resources such as the Journal/ Author Name Estimator (JANE) to help find the right journal for your paper. Choosing wisely will help your chance of success.
• Does the manuscript meet technical specifications? The staff is looking for correct formatting. Is the formatting correct? Is the manuscript close to the word count limit?
• Does the manuscript fit the submission category? For example, if the category is research, is this indeed a research paper?

2. Editor Triage

At JGME, Dr. Gail Sullivan, the editor-in-chief, reads almost all submissions and may send a manuscript to deputy or associate editors for further review. This reading is a general review, evaluating whether the topic is important, the subject fits the journal’s mission, and conclusions are generalizable. The editors consider whether the research and writing are generally well done and if a submission coincides with an accepted article. Editors often lean towards sending manuscripts for peer review. A rejection occurs when a manuscript is rejected after staff editor and/or editor review. This process usually occurs quickly, from a few days to a week. Although the feedback for desk rejection is often limited, the editors may provide some commentary; for example, the submission overlaps with a recent publication.

3. The Review

Blinded by the light

There are pros and cons to blinded review. The medical education community is comparatively small. The more familiar an individual is with the field, the more likely the individual will recognize the manuscript’s authors. Journals may ask for manuscripts to be blinded, such as removing names and references that cite the authors. However, blinding is imperfect, given that the editors ultimately know the manuscript authors and their institution(s). Even if blinding is required, sometimes seasoned reviewers in the field can identify the author(s).

One is the loneliest number

Group peer review has been discussed in JGME and Academic Medicine.^​1,2​ This involves collecting multiple perspectives, generating a team-based learning environment, dividing the workload, and providing a thorough review. A manuscript submission may include agreeing to group peer review. Thus, more individuals than previously expected may see a submission.

Becoming a Quality Peer Reviewer

What are the benefits to me as an author to participate as a reviewer?

One piece of advice for authors is to be well read. Be current with the literature and research to help you ask unique, important, and relevant questions that will advance the field. Becoming a peer reviewer, editor, or member of an editorial board helps you to stay current with the literature and research happening in your field.  

Peer reviewer training

Peer reviewers previously did not receive formal training. However, fellowships and degree programs are beginning to formalize this process. As part of a degree program, peer review methodology may be reviewed in a research design course. You may have an advisor who is willing to guide you in one-on-one mentoring in peer review process. An example of this mentorship is as follows:

1. The mentor and student read a manuscript independently.
2. After independent review, the mentor and student will compare their critiques.
3. The student will then prepare a feedback summary for mentor review. 

This process can even be carried out using a draft or already published work as an exercise to learn the process. 

Participate in group review

As mentioned above, group peer review policies may vary by journal. If you have a mentor who does peer review, ask them to write to the editor to inquire about the possibility. If approved, these mentors can perform parallel reviews with their mentee and provide feedback on the review. Once you have a solid foundation, your mentor can send your name and contact information, endorsing you as an independent peer reviewer. This helps editors and the review community expand the peer reviewer roster.

Peer review guides

The Association of American Medical Colleges (AAMC) has published a checklist in the Review Criteria for Research Manuscripts [pdf]. This checklist helps ensure comprehensive consideration. Journals like Academic Emergency Medicine send a similar guide to their reviewers. The EQUATOR Network is a collection of reporting guidelines that can provide guidance to reviewers and authors.

Never write down anything you wouldn’t want to read aloud to your mother 

Reviewers should consider signing their reviews, if permitted by journals.  This can help reviewers reconsider their tone. More junior reviewers may have concerns about career ramifications when considering whether to sign their reviews. Blinding provides a safe space for critical reviews by senior reviewers who know the authors and by junior reviewers who are early in their careers. Regardless of seniority, blinded review is never an excuse for unkind or unprofessional behavior. 

Author information

Christine T. Luo, MD PhD

Resident
Department of Emergency Medicine
The Ohio State University

| Twitter |

The post Peering into Peer Review: How can you participate in peer review? appeared first on ALiEM.