A very meta analysis: Building evidence to support evidence-based medical education
There’s plenty of evidence to show that immersive student placements deliver a lasting and solid skill-base to students, but very little evidence to explain exactly how these experiences do that. Understanding this could lead to more effective experiences.
In 2021, A/Prof Basia Diug from our Medical Education and Research Quality Unit (MERQ) – and leader of our Undergraduate Education portfolio – joined medical educational researchers from across the Monash Faculty of Medicine, Nursing and Health Sciences on a project to better understand the underlying mechanisms of scholarly experiences.
Their systematic review followed the 2020 introduction of a Scholarly Intensive Placement (SIP) in the Monash medical degree (MD).
Led by researchers at the Monash Centre for Scholarship in Health Education, the project dovetailed with MERQ’s remit to use pedagogical evidence to inform – and transform – medical education and clinical practice.
Scholarly experiences are activity-based, often immersive, experiences that are designed to provide a greater depth of knowledge in a field, the opportunity to create new knowledge, and increase capacity around self-directed learning and communication skills. They often take the form of quality improvement projects, literature reviews, and participating in working research projects.
While there has been a global movement to incorporate scholarly experiences into medical degrees, most evaluations of their success have been measured by metrics such as number of peer-reviewed publications and conference presentations generated through the experience. There has been no research exploring the complexities of how the scholarly experiences build scholarly skills, nor has there been any evidence syntheses that explore how scholarly experiences lead to a range of outcomes, and under what circumstances.
A/Prof Diug says, “Evaluating education is just notoriously difficult, and we’re always looking at novel ways to improve upon it. In this review, we tackled the questions why, how, and under which circumstances do medical student scholarly experiences produce a range of different outcomes?
“As one of the University’s leading providers of SIP placements, it’s particularly important for us as a School to be involved in this research, and pass our findings on quickly to our educational staff for translation into better teaching.”
The Monash SIP is a research-based six-week placement offered to final year medical students, giving them a deeper understanding of how the evidence-base of modern medicine is built, and a chance to consider whether medical research may play a part in their future career.
The researchers applied a realist framework called Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) to their project. Realist frameworks aim to identify and explain underlying hidden mechanisms that drive outcomes from interventions, even among complex and nuanced contextual backdrops.
They identified 28 eligible peer-reviewed papers, and synthesized their findings around context, mechanisms and outcomes. Using patterns found across the papers, they developed a unique model to explain how scholarly experiences generate results, as explained in the below diagram:
They found that scholarly experiences work by highlighting the need for students to see the value of research, and building research self-efficacy as mechanisms for achieving outcomes. The outcomes were achieved in contexts of organisations that foster a research culture, provide protected time for students, that value the scholarly experience and where students were able to exert some choice over the experience.
A/Prof Diug says, “Methodological research like this will inform the successful evolution of Monash’s SIP program, and the wider trend towards incorporating these experiences into medical education.”
The arrival of the COVID-19 pandemic disrupted the clinical rotations of Monash medical students throughout 2020, which saw the whole MD SIP program pivoted, with the School responding and supporting additional students and working around clinical placement opportunities. Efforts like this helped drive the initiative to success in its first year, and the Medical Deans of Australia and New Zealand (MDANZ) published the launch of the Monash SIP as a case study in 2021, emphasizing the value of research training experiences within the medical curriculum. In 2022, the School with the Department of General Practice will provide 90 SIPs.
“The SIP program at Monash is a huge team effort, and it’s a testament to all involved research teams and supervisors that it’s been so well received. This new research helps us understand some of the ‘whys’ behind that success.”
She’s also excited at the opportunity to drive translation of these and other methodological findings at the School’s annual Teaching Challenges and Innovation Symposium. This event provides educators within our School to learn the latest about education philosophy and design, and the incorporation of emerging technologies into the educational experience.