Analytics and assessment
Exploring the best methods to assess the quality of learning, and produce meaningful metrics.
Human-centred Teamwork Analytics
Investigators: Gasevic D, Martinez-Maldonado R, Buckingham Shum S, Elliot D, Gasevic D, Ilic D.
This project aims to develop methods to assist the assessment and improvement of collocated teamwork, by making multimodal activity traces visible and available for computational analysis. This project expects to bridge the gap between promising sensing technologies and the dearth of tools to automatically assess teamwork. Intended outcomes include co-design and modelling methodologies for human-centred analytics that map from low-level data to higher-order constructs to enable non-data science savvy users to get actionable insights into multimodal team traces. This research is highly applicable to Australia, with communication and teamwork being two of the topmost critical skills required by Australian employers.
Funding: Australian Research Council Discovery Project grant | 2021-24
Towards SDG 4: Transferring Forum Brain’s Success to Health Sciences, and Beyond
Investigators: Angus S, Ilic D, Darcy R.
Today, over 100 million people learn on a Massive Open Online Course (MOOC), and in every single course, an online discussion forum is used. Yet, no automated human-quality grading and feedback technology for open ended forums exists. ForumBrain.monash.edu, an automated, intelligent, forum analysis platform powered by cutting edge AI technologies, has recently been developed and trained by A/Prof S Angus at SoDa Laboratories in the MBS. This project takes Forum Brain to the next level by partnering with educational domain leaders and specialists in the health sciences, to test, and demonstrate, crucial cross- knowledge domain efficacy through transfer learning and beyond.
Funding: Interdisciplinary Research Support Program | 2019
Defining and assessing the utility of assessment categories in health professional education
Investigators: Ilic D, Diug B, Maloney S, Mundy M, Davis E, Gibson S, Sarkar M, McKenzie S, Mak V, Darcy R.
Assessment is a key component of any health professions degree. Robust assessment processes are fundamental in ensuring graduates practice safely and professionally while optimising health outcomes of patients and populations. The overall aim of this study is to create, apply and test consistent definitions and categories for assessment types (i.e. an assessment framework) in the health professions, in the Faculty of Medicine, Nursing and Health Sciences (FMNHS).
Funding: Learning & Teaching Grant, Faculty of Medicine, Nursing and Health Sciences | 2019
Psychometric validation of a tool used in the assessment of professionalism for pre-clinical medical students
Investigators: McArdle A, Karim N, Tara M, Reser D, Ilic D, Barton P, Presley M.
Professionalism, defined as “the ability to meet the relationship-centred expectation required to practice medicine competently”, is a core competency for medical graduates. Lapses in professionalism may risk patient’s health and lead to disciplinary action. Development of an objective, fair, consistent, accurate, and robust means of assessing professionalism beginning from the earliest stages of training will maximise opportunities for Monash medical graduates to develop critical communication and inter-personal skills that underpin success in healthcare delivery. This project aims to develop and psycho-metrically validate a professionalism assessment tool for pre-clinical medical students.
Funding: Monash Education Academy | 2019
Research- and practice-based development of a framework and guidelines for Assessable Online Discussion (AOD)
Investigators: Darcy R, Ilic D, Gasevic D.
Online discussion forums have become a key pedagogical component of online education. Participation in online discussions is more effective if it is linked to assessment. There is ongoing debate, however, on what should be assessed in online discussions, and how that assessment should be implemented. This project will undertake a systematic review of literature in regards to assessable online discussion (AOD), and investigate the use of AOD in units within the Master of Public Health courses. The results of the review and investigation will be used to develop a framework and guidelines for AOD in the MPH, with the ultimate goal to develop a model for best practice AOD that potentially will be incorporated into Faculty and University policies and guidelines.
Funding: Monash Education Academy (MEA) | 2018
Development of an Objective Structured Clinical Exam (OSCE) for internationally qualified nurses
Investigators: Griffiths D, Brooks I, Endacott R, Dix S, Rees-Sidhu C, Barton P, Hall H, Gibson S, Harrison J, Ilic D, Maloney S, Leech M, Adams K, Merriman C, Fuller R, Homer M, Vance S.
This project on developing and evaluating a series of objective structured clinical examinations (OSCEs) for the assessment of internationally qualified nurses. All aspects of assessment were examined including blueprinting, station development, psychometric and cost analysis.
Funding: AHPRA | 2017
Developing psychometrically validated tools for assessing medical students' competency in research knowledge and skills
Investigators: Ilic D, Diug B, Campbell D, Sadasivan S, Bell R.
The current MBBS degree is in transition to a MD in 2020. A key aspect required for the current degree to meet the Australian Qualifications Framework (AQF) for a MD is the presence of specialized knowledge and skills in research. This current proposal aims to develop a suite of psychometrically validated tools that will assess MBBS student competency in research knowledge and skills. This will provide the Faculty with evidence to demonstrate student competency at the AQF level 9, which is currently lacking in the MBBS degree and a key requirement for its transformation into a MD.
Funding: MBBS Development Grant Scheme | 2015