Analysis of the ACT Matrix for its Suitability for Identifying Diverse User Needs of Online Exam System

Universities have started shifting from paper-based to online exams in recent years. Taking exams online offers the opportunity to automate the assessment - a real leg up for all teachers. However, online exams depart from a well-understood century-old paradigm, potentially violating human values such as safety, privacy or self-determination to the effect that students may not perform to their full potential. Hence it is paramount to minimise value violations in an online exam system.

The question that we tried to answer with this research is whether the application of the ACT Matrix - an instrument from the acceptance and commitment theory in the field of psychology - is suitable to take the first step: the successful identification and creation of an in-depth understanding of individual value violations that students experience while taking online exams.

The ACT Matrix is a self-assessment tool that allows the exploration of one’s own values and violations, as well as their links to the behaviours a person may show. By doing so, the ACT Matrix supports the creation of a map that links cause and effect, and thus helps to gain a deeper understanding of the diverse needs of student users of an online exam system.

This project used the Monash eExam system as a case study. We surveyed students during several exam periods in 2022. Students were asked to reflect on their most recent exam experience with respect to their values, violations and related behaviours. Moreover, students were asked to link elements of their ACT Matrices to user stories to describe improvement requests for the eExam system. Data was collected with a combined online ACT Matrix - user story template instrument. The instrument included instruction videos that guided students through the survey.

Preliminary results indicate that the ACT Matrix is a suitable tool for the analysis of end user needs but that its administration poses challenges. Further research is required to substantiate our findings.

This project is now in its final stages. We currently work on the completion of the analysis of the data and the synthesis of initial insights. Our goal is to report our findings with a scientific paper.

Project Lead

Dr Ingo Mueller

Project Team

Dr Waqar Hussain (Data61), Paul McIntosh (eSolutions)