Automated Evaluation of Readability for Moodle (LMS) courses using the Robot Framework
Monash University uses the Moodle Learning Management System (LMS) to facilitate the online delivery of learning content to over 85,000 local and international students annually. Student feedback has revealed usability issues with Moodle which impact the learning experience. One reason may be the diversity of our students who, for example, have varying cultural backgrounds, language skills or previous training. Another reason may be the lack of built-in tools that enable teaching staff to tailor their online content to the needs of a diverse group of students.
This research aimed to (a) determine the extent to which the readability of text can be automatically evaluated on Moodle and (b) what information can be drawn from a calculated readability score. Readability is concerned with how well a reader comprehends a written text. A readability score is an index that indicates the level of education required to fully comprehend a given text.
A team of four FIT4003 students started with a review of several readability metrics including Flesch Reading Ease, Flesch-Kincaid, Gunning-Fog and SMOG. In a second step, the students created a tool prototype for the calculation of readability scores based on available free and open-source software including Python Readability and Selenium text segmentation libraries and the Robot-Testing Framework.
The tool automates (i) the extraction of text from Moodle components, (ii) the separation of individual text segments, and (iii) the analysis and score calculation for these segments. It was tested with selected Faculty of IT (FIT) Moodle units. The calculated readability scores were further evaluated with student feedback collected via an online survey.
The outcome of the evaluation showed that human and tool scores differ significantly depending on the text under assessment. Moreover, 75% of survey responses indicated two major issues that affected text readability: (i) required prerequisite knowledge and (ii) required context. Less frequently reported readability issues included (i) used technical jargon and (ii) sentence length.
Project Lead
Dr Ingo Mueller, Paul McIntosh
Project Team
FIT 4003 student team
