As part of the ALiEM Faculty Incubator Program, Dr. Mike Callaham (Editor-in-Chief of Annals of Emergency Medicine) and Dr. Ellen Weber (Editor-in-Chief of Emergency Medicine Journal) participated in a Google Hangout where they provided expert advice on academic writing and peer review. We have summarized their wisdom below.
Google Hangout Video Discussion
Timestamps for the Google Hangout
00:40 – How did you become an editor of a major journal?
03:00 – How do you work from becoming a reviewer to an editor?
07:50 – Skills acquired by performing peer review
11:20 – Biggest pitfalls for new authors
15:00 – Have empathy for the reviewer, comments aren’t personal
18:20 – Mike advocates for submission of manuscripts for all abstracts
19:40 – Don’t oversell your conclusions! Even negative articles are worth publishing
21:50 – Replication crisis
22:50 – Methods/Results >> Discussion/Introduction
25:00 – What to do about mean or incorrect reviews?
27:00 – Maybe it is your fault; How to respond to reviews
30:15 – Advice for new reviewers
33:35 – Peer review is like English composition
35:40 – Checklists from Annals and EMJ
37:00 – Predatory journal watch
41:30 – The future of medical scholarship
46:42 – Final message from Mike for EM educational researchers
49:25 – Final message from Ellen for EM educational researchers
52:30 – Wrap Up, Main Highlights
[/fusion_toggle][/fusion_accordion]
Shorter podcast version
Why Peer Review?
Reviewing is a critical skill for academicians. Reviewing, over time, makes us better clinicians, educators, researchers and critical thinkers. It also hones mentorship skills. The more you review, the better you can guide mentees in research and writing.1,2
The goal of peer review is to improve the paper and mentor the author. Keep this in mind when reviewing papers. Sarcasm and flippancy don’t help anything.
Getting Started as a Reviewer
Don’t sell yourself short as a reviewer. Some of the best reviewers have been those early in their career. We all have expertise in lectures we do, innovations we create and research we’ve performed. This can be the foundation of a career in reviewing despite a lack of granular methodological knowledge. Often, a reviewer knows the topic and literature on that topic better than the editor and can give “common sense” insight, which is valuable to the readership. Often it is the simple big picture that is the pitfall of the research, rather than the detailed statistics. Answer the question: will this paper impact my practice?
- Workshops don’t improve peer reviewing skills1,2
- Does mentoring new peer reviewers improve review quality? A randomized trial.3
- Annals of EM’s Peer Review Training Module
Showing up is more than half the battle. Many opportunities arise purely from taking the time to respond to requests, showing up at meetings and persistence in participation. Doors will be opened and you will be sought out for positions if you take your responsibilities seriously and perform well. You need to demonstrate your skills and motivation. Editors notice and often reward a thoughtful/insightful/productive and punctual reviewer.
Keep trying. It takes time to succeed, but if you put in your time and remain active, editors will put in their time to help develop your abilities. Take all opportunities to speak with editors and reviewers if this is something that interests you. If you review an accepted manuscript, read all the other reviews when they are sent to you. If there are points that are made that you didn’t even see, use these opportunities to learn.
Be professional in your feedback as a reviewer and your response as a writer. Reviewers should show the same grace we show towards patients with newly diagnosed disease: with humility and nuance.
Writing Tips from Editors-in-Chief
New author pitfall: Not knowing the research question from the beginning. You have to know your research question and hypothesis before starting a project. Reviewers should be able to predict your methods from your background. Who are your subjects? What are your outcomes? Utilize a research checklist in your preparation.
New author advice: Tell a story that explains your research. Finding tangents and subgroup analyses can lead to statistical significance but a confusing, unfocused and meandering manuscript can sink even good projects.
The discussion should never be longer than the results. Don’t write pages of discussion and introduction, the meat of the paper should be the methods and the results. The discussion should be obvious based on the content of the methods and results, which should be the two biggest sections of any manuscript.
Dealing with Rejection: Have a “growth mindset” regarding rejection. If you receive a rejection/request for revision, take a week after reading it initially to respond. The emotional attachment to your manuscript can cloud the response of even hyper-rational beings. Reviewers often give quite insightful feedback that can make your manuscript better. Take advantage of this advice and improve your manuscript. And submit again. And again. And then use the feedback to improve your next project.
A rejection does not mean your research is not publishable.4 Rejections can occur to even great projects if they are sent to inappropriate or less relevant journals. Research the journal prior to submission to select the appropriate home for your project. Many papers are initially rejected. Utilize reviewer/editor feedback to make it better and submit again.
It’s always reasonable to disagree with reviewers. Many times it’s as simple as rewriting the passage. Do it with humility and address the reviewers concern while explaining your side of the issue. This works more than you’d think. Address EVERY comment in your response.
Publication Tips and Tricks
Negative studies are still valid research.5 Specific outcomes shouldn’t matter if the question is good and the methods are sound. Errors and dead-ends in research can actually lead to advancement of science. Don’t be discouraged by negative results. They can be more meaningful and change science more frequently than positive results.
It is our scientific and ethical obligation to attempt to publish abstracts. All abstracts have a 50% publication rate.5 We owe it to the patients involved to bring these to publication as abstracts are not really peer-reviewed projects.
Be wary of predatory journals. If you are receiving requests for submission or editing, especially from a field you aren’t studying, this is highly likely a predatory journal. If there is a fee to submit to a journal (not for publishing), this is highly likely a predatory journal. Some will take your money and not publish your research.
Author guides
- Annals of Emergency Medicine
- Emergency Medicine Journal
- Western Journal of Emergency Medicine
- Journal of Emergency Medicine
Additional Reading
Callaham M, Wears RL, Weber E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002 Jun 5;287(21):2847-50. PMID: 12038930
Author information
The post Interview with Drs. Michael Callaham and Ellen Weber: Behind the scenes of journal peer review appeared first on ALiEM.