The Case of the Catastrophic Classroom outlined a scenario where a junior faculty member is tasked with revamping didactics at her institution. We joined Jill as she walked through various phases of discovery, building empathy for her stakeholders. This case was subsequently discussed at the CORD Academic Assembly 2016 (#CORDaa16) where 4 teams competed to design novel solutions for this complicated problem.
This month the ALIEM Design team, led by Drs. Teresa Chan (@TChanMD), Catherine Patocka (@patockaem), Jeremy Voros (@vorosmd) co-hosted a design challenge with CORD’s Dr. Rob Cooney (@EMeducation) where a keen bunch of creative medical educators participated to identify the problems and pitch possible solutions that might work for Jill. Their discussion and solutions were based on the insights and suggestions from the ALiEM community. We are proud to present to you the Curated Community Commentary and our Design Hackathon team solutions. Thank-you to all our participants for contributing to the very rich discussions last week.
This follow-up post includes
- A special summary of the ALiEM-CORD Design Hackathon, detailing the method that our team used to facilitate the problem-solving of our participants
- A summary of the 4 groups’ suggestions, and an account from one of the participants (Dr. Therese Mead).
- A summary of insights from the ALiEM community derived from the Twitter and blog discussions
- Freely downloadable PDF versions of the case and expert responses for use in continuing medical education activities
[su_spoiler title=”Why run a Design Hackathon? by Dr. Rob Cooney ” style=”fancy” icon=”chevron-circle”]
Why ‘hack’ conference?
Simon Sinek says to ‘Start with Why?’ So why did we want to ‘hack’ conference? Quite simply, this was a way to share our passion for education, demonstrate the principles of design thinking, and stretch our own abilities. It all started with an idea. Each year, as the CORD Academic Assembly winds down, the track chairs meet to discuss what worked well, what needs the ax, and what new ideas we can come up with for the following year. The 2015 meeting was a resounding success and pushed the #FOAMed sessions into the limelight. It was so popular, in fact, that we earned our own track, though renamed iMedEd (Innovations in Medical Education). Not wanting to rest on our laurels, we proposed several new ideas.
I had been studying quality improvement at the time and design thinking kept coming across my radar (the tools of QI and Design Thinking have significant overlap). We decided that Design Thinking definitely fit within our scope of innovative practices, but how to apply the tenets? How about a design thinking workshop! Thus, the Hack-a-thon was born. Needing a LOT of assistance, I reached out to Michelle Lin and Teresa Chan who readily agreed to help out and the rest is now history.
What is Design Thinking?
Design thinking (DT) is a mindset that can help propel innovation. DT proposes an integrative approach to innovation by focusing on three overlapping ‘stages’: discovery, ideation, and implementation. The process is NOT linear. Like the PDSA cycles (Plan, Do, Study Act) of QI, DT proposes using an iterative process to refine each phase and maximize learning. While engaging in this process, design thinkers must also manage the relationship between feasibility, viability, and desirability. Feasibility represents what is actually possible, viability focuses on the business model, and desirability represents what people will accept or seek out. Design thinking can be used to improve many vexing problems in medical education, whether the problem relates to the curriculum, learning environment, processes, and even the education “system.”
How Does Design Thinking Work?
Part 1: Discovery
Design thinkers begin the process with problem identification. This is often a very easy task. Just think to yourself about problems, frustrations, and other things in your life that need to be ‘improved’. Any of these are fair game for DT.
HINT: Carry note cards around with you to capture these issues and create a ‘bug list’ to address in the future!
Part 2: Interpretation
Once the problem is identified, design thinkers then engage in defining the problem. DT is ‘human-centered’ meaning that defining the problem involves the development of deep empathy with ‘end users’. In medical education, these could be learners, teachers, and even the patients. This stage involves observing how people interact with the problem and in depth interviewing people to gain their insights. Refining the problem can take several interviews to get to the bottom on the core problems.
HINT: Use a structured interview questionnaire to make sure that everything is covered, but don’t be afraid to use a LOT of clarifying questions (tell me more; why?).
DT is a collaborative skill set and encourages groups to work together to refine the problem. Reviewing the data collected above as a group can help parse out the key needs and ideas.
HINT: It’s Post-It time! As you review the data, try to put one key insight on each sticky note. Then, as a group, begin to sort the notes into themes and subthemes. How many different ideas can you identify?
Part 3: Ideation
With the problem is defined, it’s time to ideate. “How might we…?” Is a simple way to create actionable ideas. These questions help to promote brainstorming. As a group, get the sticky notes out again and see how many possible solutions you can create to answer the question. The key here is not judging anything and building on others ideas. “Yes, and…” and a fun way to add to others ideas. For instance a question you might ask with this past month’s case would be:
“How might we…improve resident lectures?”
A colleague might respond:
“We could do a slide design workshop”
And then you might say:
“Yes! And we could award prizes for the most novel slides.”
After the brainstorming process wraps up, you should begin to see patterns in the ideas. As a team, engage in a note sort similar to the problem identification stage. This should provide you with several possible solutions to your identified problem. If not, keep brainstorming!
HINT: Institute some rules for brainstorming to help promote creativity. We like IDEO’s brainstorming rules.
HINT: If you get stuck, there are tools that can prompt you. Examples include: OFlow app, IDEO Method Cards, Unstuck Cards, ThinkPack.
Part 4: Experimentation / Prototyping
“Fail fast, fail often, to succeed sooner.”
-Entrepreneurs mantra
The next phase of DT is implementation. It is important to recognize that you can not just take your brainstorming solution and put in action. Instead, this phase begins with experimentation.
Failure will be a constant companion early in the experimentation phase. This makes it important to prototype. Prototyping involves actually diving in and putting an model solution together and trying it out. In educational prototyping, this can be accomplished by creating a storyboard, diagramming the proposed solution, mocking up the proposed application or learning space, or even creating a role play to demonstrate the solution.
Also critical to this phase is feedback. Take your prototype and show it off to your learners and faculty. Their feedback will unveil flaws in the prototype and allow you to fix them during this phase. The earlier you learn these, the sooner you can move from prototype to launch.
[/su_spoiler]
[su_spoiler title=”How to run your OWN design challenge. By Dr. Rob Cooney” style=”fancy” icon=”chevron-circle”]
What do I need to get started?
Hosting a design challenge is a great way to learn the process and expose learners to the concepts. Before you begin, identify a problem that you want the participants to focus on solving. We used the weekly didactic experience as our starting point. Try to find a room that allows for space and movement, gather your supplies, and start designing!
To make things easier, the following two files are available for you to download and use as a sample template or activity booklet. This is what we used for the ALiEM CORD Design Hackathon.
Click to view the iMedEd Hackathon slideset
Click to view the iMedEd Hackathon handout
Here is a description of what we did though…
Discovery: At CORD, we decided to accelerate the empathy process by putting up the Catastrophic Classroom case to curate a community response to the problem. We also recruited faculty and residents to serve as end users. The participants interviewed them to gain insights into the problem.
Interpretation: From here, the participants moved to the Post-Its followed by a large group note sort.
Ideation: Teams formed, and people wrote a very specific problem statement. From here, the teams engaged in brainstorming followed by another large note sort.
Experimentation: Teams took their ideas home overnight to prototype. They were tasked with showing their solutions to fellow faculty members for feedback and integration. After refining their idea for a final hour, they presented their ideas to the judges, many of whom were end users from Day 1.
Recommended supplies
- Post-It notes (lots and lots)
- Markers
- Camera (phones will work)
- White boards (you can purchase cheap whiteboard panels at your local hardware store) – Alternatively, you could buy the large 3M Post-It Easel Pads).
- Tape
- Music
- Candy
- Timer (search google for “google timer” and a nice web-based stopwatch can be displayed)
Optional prototyping supplies
- String
- Popsicle sticks
- Cardboard/posterboard
- Rubber bands
- Scissors
Want to learn more?
[/su_spoiler]
[su_spoiler title=”#CORDaa16 Participants’ Experience by Dr. Therese Mead ” style=”fancy” icon=”chevron-circle”]
When I registered for the 2016 CORD Academic Assembly, I was excited to discover the iMedEd track. I immediately signed up for innovative discussions ranging from slide design to podcasting. I can’t believe that I almost missed the best experience of iMedEd– a 2-day Design Thinking Hackathon! Although I didn’t register for the event (yes, I’m calling myself out on this one), I was having a great time and learning so much from the other iMedEd sessions that I walked in at the beginning of the Hackathon to see what was going on. I expected to sit on the sidelines, but Dr. Teresa Chan invited me to join a small group and the next thing I knew, I was fully immersed in a Design Thinking challenge.
This was my first experience in Design Thinking and gave me a whole new outlook on difficult problem solving. Initially, I sat at a table with a group of 5 of my fellow EM educators and we interviewed Jill’s residents and peers to define the problems with her educational conferences. After a brief brainstorming session, we collaborated with the rest of the groups in the Hackathon to organize the problems we identified (visually displayed in a rainbow of sticky notes on project boards).
For the next part of our challenge, we identified themes to the problems, then divided into small groups to start exploring solutions. I teamed up with Dr. Michael Overbeck to form Team M&M. Team M&M tackled the issue of quality of Jill’s conferences and we spent the next day developing and prototyping potential solutions. (More on this below!)
The guidance of the facilitators really made the experience. I was encouraged to step out of my comfort zone and consider our task in a very organized manner that promoted creativity and collaboration. We were ready to pitch a promising solution to the judges by the end of the 2 day event – thanks to the Hackathon team. I’ll definitely be taking Design Thinking back to my residents and can’t wait to put my new skill into action.
[/su_spoiler]
[su_spoiler title=”#CORDaa16 Participants’ Case Solutions by Dr. Jeremy Voros” style=”fancy” icon=”chevron-circle”]
All 4 of the Design Teams did extremely well in the Design Challenge. At the end, a mixed panel of faculty and residents helped us to select our winners… However, in our hearts they all had great, winning ideas!
First Place: Team Engagement: “All In”
Alexandra Grino, Jordan Spector, Jessie Nelson, Leslie Oyama, Antonia Quinn
Their Selected Problem
The EM faculty have little incentive to attend and participate in the division’s weekly didactic curriculum
Their Solution
Team All-In developed a gamification of conference called “The Conference Cup” – a novel award for the faculty team that contributes most to the didactic curriculum, scored by “EVU’s”, an academic analog to the clinical value of RVU’s. Varying levels of participation by residents and faculty earn their team varying numbers of EVUs. A publicly displayed leader board, and a yearly draft of faculty team members increase the stakes for active participation. Additionally, the highest scoring member in the winning team is awarded an individual prize.
Second Place: Team M&M
Team members Therese Mead and Mike Overbeck
Their Selected Problem
Lack of faculty engagement, inconsistent lectures, and decreased resident interest lead to poor quality experiences for all participants.
Their Solution
Team M&M also converged on improving the lecture experience by applying gamification to the lecture experience. In their solution, residents and faculty will be placed on teams and points will be assigned for desirable accomplishments and behaviors (e.g. innovation, application of novel instructional methods, resident and faculty participation, completion of asynchronous exercises, quality of faculty lectures). By increasing the stakes and intensity of involvement of all participants, they would improve the lecture quality and enhance the quality of the synchronous didactic experience.
Third Place: Team MedEx Delivery
Team members Aaron Brown, Bob Sobehart, Marc Phan, and Chris Fowler
Their Selected Problem
Lack of innovative educational techniques in resident education
Their Solution
Team MedEx gave what was surely the most theatrical pitch, proposing to increase use of innovative teaching methods in resident education by encouraging bi-directional mentorship and team teaching, improving familiarity and access to progressive educational resources, and providing positive incentive for use of these methods. A most feasible solution!
Fourth Place: Team Couch
Team members Ankit Shah, Loice Swicher, Daniel Robinson, Zach Jarou, and Shana Nelson
Their Selected Problem
St. Elsewhere’s conference suffers from a toxic learning environment.
Their Solution
Team Couch proposed quarterly retreats with residents and core educational faculty meant only to focus on the learning experience, rather than learning emergency medicine. With time set aside for socializing and sharing between key stakeholders, with a little liquid courage facilitating, the residency will move forward together to improve their conference.
[/su_spoiler]
[su_spoiler title=”The Case of the Catastrophic Classroom: Curated from the community – by Dr. Catherine Patocka” style=”fancy” icon=”chevron-circle”]
This month’s case focused on Jill, a junior faculty member who has recently been hired to an education leadership role at St. Elsewhere hospital. Her first challenge in this new role is to revamp their weekly half-day educational conference. She is faced with an uninterested overworked Program Director, faculty members who rarely attend conference and when they do don’t engage because a lack of incentives (CME or financial), and instructional design that focuses largely on didactic lecture presentations and is primarily senior resident driven. Jill reaches out to her mentors from the CORD listserv and receives a mixture of advice ranging from FOAMed resources, to varied instructional strategies (flipped classroom, small group sessions, self-directed study), but others caution about maintaining ACGME compliance.
Problems with EM conference identified by commentators were similar to those in the case: specifically that improving EM conference is an extremely challenging task, lecture-based instructional strategies are generally less desirable than other active learning strategies and an almost universal difficulty with faculty engagement and participation.
With regards to solutions a few major themes arose from the discussion, in particular the importance of engaging stakeholders as well as participants in the solution, considering the use of a variety of instructional strategies and considering whether the ACGME requirements need to be updated to reflect a more learner-centered competency-based approach.
Several discussion participants outlined their experience with trying to single-handedly raise the level of conference. Dr. Jordan Spector described “biting off more than he could chew” by simultaneously delivering lectures, revamping journal club and overseeing/managing the entire curriculum, while Dr. Anand Swaminathan noticed that by giving weekly lectures to the residents his colleagues began to loathe him as the residents began to replicate his behaviors/practice. Dr. Will Goldenberg suggested changing a small subset of conference at a time to harness progressive buy-in and support of change. Most commentators felt there was a strong need for balance and accountability with regards to quality improvement and there was strong consensus on the need for support from the program director, department chair, and other faculty. With regards to solutions, Dr. Aaron Brown highlighted the need to cultivate a leadership team and core faculty group to create the energy excitement and support necessary to drive change. One might also consider developing a Conference Improvement Committee with faculty and resident members to harness a culture of ownership and accountability and invest in some faculty development for those that are involved in EM conference.
There was strong consensus on the challenges of a primarily lecture-based format. Dr. Jeff Riddell pointed out lecture-based strategies do not promote active learning, do not capitalize on or address a variety of learners, and learning styles and can lead to faculty repeating the same content year after year. Despite these challenges it can be difficult to incentivize faculty members to change the way they deliver educational material. Commentators suggested a variety of alternative instructional strategies that could be used in place of lectures including:
- Small group sessions
- Simulation sessions
- Flipped classroom (or blended learning)
Our participants generally stressed that implementation of these strategies was contingent upon a strong faculty development program to provide faculty members the time and training necessary to practice and implement these new strategies. Dr. Loice Swisher brought up the incorporation of independent individualized instruction (asynchronous learning) as a way of exposing residents to some of the best teachers/lectures that EM has to offer. She suggested lectures from regional and national EM conferences or the ALiEM AIR Series which introduces residents to some of the best FOAM available.
A few less dramatic (potentially more feasible) changes included Dr. Riddell’s suggestion of including core content lectures delivered in 15-20 minutes followed by break out small group discussions stratified by training year and Medical Education Fellow Dr. Alex Chorley’s suggestion to incorporate pre/post lecture quizzes to promote interest in the topic and require active recall of the material. Dr. Will Sanderson suggested reorganizing morbidity and mortality conferences towards using mistakes as opportunities for growth and quality improvement. Additionally Dr. Kory London recommended incentivizing faculty using the CORD supported “Core Faculty” designation and ensuring that all of the conference content is not resident-driven. Overall commentators, advocated the application of solid learning principles to whatever approach is taken, and Dr. Rob Rogers suggested the book Make it Stick as a good starting point.
Finally, many commentators were cognizant of Accreditation Council for Graduate Medical Education (ACGME) accreditation standards and advocated a balance of large-group planned educational activities and independent individualized instruction to ensure standards were met. Drs. Riddell and Sanderson advocated for revision of the ACGME standards to reflect a more learner-centered competency based approach questioning why every learner (regardless of their ability and performance) has the same attendance requirements despite conference attendance not being correlated with in-training exam scores (1).
References
- Hern Jr, Gene, et al. Conference Attendance Does Not Correlate With Emergency Medicine Residency In‐Training Examination Scores. Academic Emergency Medicine. 2009 (16 Suppl 2): S63-S66. PMID: 20053214.
[/su_spoiler]
Case and Responses for Download
Click Here (or on the picture below) to download the case and responses as a PDF.
Author information
The post MEdIC Series | The Case of the Catastrophic Classroom – Expert Review and Curated Commentary appeared first on ALiEM.