The use of blogs and podcasts within health professions education is rapidly increasing, especially among emergency medicine and critical care learners [1-5]. However, there are no standardized quality assessment methods for the learners and educators that use and produce them. This dilemma led the MedEdLife Research Collaborative to launch a research agenda with the goal of developing a tool to assess the quality of blogs and podcasts. This was done through the series of studies that are presented in this blog.
Study 1:
Paterson Q, Thoma B, Milne WK, Lin M, Chan TM. A Systematic Review and Qualitative Analysis to Determine Quality Indicators for Health Professions Education Blogs and Podcasts. J Grad Med Educ. (in press). DOI: JGME-D-14-00728.1 [6]
A systematic review of the literature was used to identify quality indicators that have been applied to other secondary educational resources. Lead author Quinten Paterson noted that:
“It was incredible to see all of the potential quality indicators that could be used to assess a resource.”
These quality indicators then underwent a qualitative analysis to categorize and theme the results and to determine which potential indicators of quality that could be applied to health professions blogs and podcasts. A focus group of bloggers and podcasters identified additional quality indicators relevant to blogs and podcasts that were not found in the literature. Ultimately, a list of 151 quality indicators was developed across three main categories:
- Credibility (n=53)
- Content (n=44)
- Design (n=54)
This list, while comprehensive, was too lengthy to be of practical significance to producers, editors, users, and researchers. As such, our team next tried to determine how this list could be further refined.
Study 2:
Thoma B, Chan TM, Paterson QS, Milne WK, Sanders JL, Lin M. Emergency medicine and critical care blogs and podcasts: establishing an international consensus on quality. Ann Emerg Med. 2015. PMID: 25840846 [7]
The quality indicators from the previous study formed the basis of a modified Delphi process. This consensus-building study used two iterative surveys to identify the quality indicators most important for blogs and/or podcasts. Content producers (established bloggers/podcasters on the Social Media Index [5]) served as experts. The initial 151 quality indicators were narrowed to items that reached ≥90% consensus within the group:
- 14 quality indicators for blogs
- 26 quality indicators for podcasts
“This study is the first to assess the relative importance of blog and podcast features for quality. The results have already influenced the content I produce at BoringEM and ALiEM. I hope that it gives other content producers ideas for how they can make the great resources that they are producing even better.” – Dr. Brent Thoma
As content producers (bloggers and podcasters) could be perceived as a biased audience for determining quality, our next step was to approach traditional medical educators for their input.
Study 3:
Lin M, Thoma B, Trueger NS, Ankel F, Sherbino J, Chan T. Quality indicators for blogs and podcasts used in medical education: modified Delphi consensus recommendations by an international cohort of health professions educators. Postgrad Med J. 2015 PMID: 26275428 [8]
In this study, a population of expert medical educators participated in a second modified Delphi process. The expert group of participants was composed of attendees of the 2014 International Conference on Residency Education’s (ICRE) Social Media Summit. The educator community identified a short list of 13 quality indicators that reached ≥90% consensus:
- 3 quality indicators for blogs
- 1 quality indicator for podcasts
- 9 quality indicators for both blogs and podcasts
“This study involving the educator community was a key step in my mind towards building a legitimate list of consensus recommendations about what comprises quality and trustworthiness for blogs and podcasts. I hope this is a major first step towards legitimizing blogs and podcasts as legitimate educational resources and academic scholarship in the health professions community.” – Dr. Michelle Lin
Quality Checklists for Blogs and Podcasts
The results of the two Delphi studies were analyzed and amalgamated to form the Quality Checklist for Blogs [PDF] and the Quality Checklist for Podcasts [PDF].
This marks the first in a planned series of knowledge translation products that serve to help end-users operationalize this research. These analogous, platform-specific tools display the quality indicators that were identified as being of utmost importance in the appraisal of online health professions blogs and podcasts by expert populations of bloggers/podcasters and medical educators. We believe that these resources may benefit researchers and three key populations:
- Producers: who can use these items as a guide to ensure they are producing content of the highest quality possible
- Editors and curators: who can apply these checklists to the resources they are assessing before disseminating them to end-users
- Users: who can utilize these tools to help determine if the resources are of good quality.
Admittedly, the present studies have not derived these insights from the end-user perspective. Further research is underway to examine this critical issue.
Conclusion
We hope you all find these checklists helpful. We have done our best to develop them using a rigorous process based on expert consensus and are still working to improve them. We would love to hear your feedback! Please provide it by following this link and reviewing their formal publication on The Winnower. They are also available for download by following the links below.
Note: Please do NOT cite this blog post to reference these checklists. Rather, please cite their formal publication from The Winnower [9]. They can be cited as:
[su_heading]
Colmers IN, Paterson QS, Lin M, Thoma B, Chan TM. The quality checklists for health professions blogs and podcasts. The Winnower. 2:e144720.08769 (2015). DOI: 10.15200/winn.144720.08769
[/su_heading]
References
- Cadogan M, Thoma B, Chan TM, Lin M. Free Open Access Meducation (FOAM): the rise of emergency medicine and critical care blogs and podcasts (2002-2013). Emerg Med J. 2014; 31(e1): e76-7. PMID: 24554447
- Loeb S, Bayne CE, Frey C, et al. Use of social media in urology: data from the American Urological Association (AUA). BJU Int. 2014; 113(6): 993-8. PMID: 24274744
- Mallin M, Schlein S, Doctor S, Stroud S, Dawson M, Fix M. A survey of the current utilization of asynchronous education among emergency medicine residents in the United States. Acad Med. 2014; 89(4): 598-601. PMID: 24556776
- Purdy E, Thoma B, Bednarczyk J, Migneault D, Sherbino J. The use of free online educational resources by Canadian emergency medicine residents and program directors. CJEM. 2015; 17(2): 101-6. PMID: 25927253
- Thoma B, Chan T, Benitez J, Lin M. Educational Scholarship in the Digital Age: A Scoping Review and Analysis of Scholarly Products. The Winnower. 2014; 2:e141827.77297. DOI: 10.15200/winn.141827.77297
- Paterson QS, Thoma B, Milne WK, Lin M, Chan TM. A Systematic Review and Qualitative Analysis to Determine Quality Indicators for Health Professions Education Blogs and Podcasts. Journal of Graduate Medical Education. In-Press; doi: JGME-D-14-00728.1
- Thoma B, Chan TM, Paterson QS, Milne WK, Sanders JL, Lin M. Emergency Medicine and Critical Care Blogs and Podcasts: Establishing an International Consensus on Quality. Ann Emerg Med. 2015; 66(4): 396-402.e4. PMID: 25840846
- Lin M, Thoma B, Trueger NS, Ankel F, Sherbino J, Chan T. Quality indicators for blogs and podcasts used in medical education: modified Delphi consensus recommendations by an international cohort of health professions educators. Postgrad Med J. 2015; PMID: 26275428
- Colmers IN, Paterson QS, Lin M, Thoma B, Chan TM. The Quality checklists for health professions blogs and podcasts. The Winnower. 2:e144720.08769 (2015). DOI: 10.15200/winn.144720.08769
Author information
The post The quality checklists for health professions blogs and podcasts appeared first on ALiEM.