Ensure consistency and quality in marking
Chief Examiners must put processes in place to ensure assessments are marked fairly and consistently across all markers, modes and teaching locations. In addition to following the steps for valid and purposeful assessment and building effective marking schemes and rubrics, this can be achieved through moderation practices.
Moderation is the process of regulating the marking of individual assessors to achieve consistency in the application of assessment criteria and performance standards. (Morgan et al., 2004). Moderation allows markers to compare their judgements and establish a shared understanding of levels or performance and student achievement. Even where there is only one marker, it is important to follow quality assurance processes to ensure consistency in judgement and awarding of marks.
The below activities can support quality and consistency in marking:
- Ensure that marking tools and resources are developed and provided to markers
- Develop robust assessment instruments (e.g., rubrics)
- Involve markers in the design of tasks and marking schemes or rubrics to ensure a common understanding of performance standards
- Prepare sample answers or past submissions to use as a calibration exercise, establishing consensus amongst markers prior to commencement.
- Discuss the meaning of standards that might be open to interpretation (e.g., the scope of terms such as ‘extensive’ or ‘broad’) and update the marking guides or instructions
- Discuss and agree on the style and length of feedback to be given, including the quality and depth
- Allocate markers according to expertise and strengths. It can be useful to ‘pair’ experience and inexperienced markers together or allocate markers per question rather than per submission
- Ensure processes are in place to minimise the risk of unintentional bias in marking such as double marking, panel marking or second marking. This should be implemented when blind marking is not practicable and when the unit's chief examiner is involved in the assessment design.
- Begin marking by discussing the marking criteria or rubric and how to interpret it
- Where possible, run a calibration activity using a few submissions or samples
- Regularly engage in discussion about standards and performance to ensure consistency of marking (e.g., borderline grades)
- Exercise self-moderation (markers can be inconsistent with their own marking, especially if there are many assessments to mark and they are marked over several marking sessions. Check back over scripts that have been marked to ensure that scripts marked earlier and later have been treated similarly)
- Second-marking, Double-marking, sample marking and blind marking processes that have been decided on should be used to review consistency of judgement
- Include one or more ‘control’ papers throughout the marking process, to either enable comparison of different markers (blind) or to ‘recalibrate’ understanding
- Double marking (blind) of fail, borderline, difficult or conflicting grades
- Review of borderline assessment tasks, High Distinctions and Fail papers to ensure grades reflect academic standards
- Review of grades and marks to identify and investigate inconsistencies and make adjustments for marker inconsistency or assessment tasks that did not appropriately assess the unit learning outcomes or were too difficult. Marks can only be scaled when a review of student performance on an assessment task, or part of it, indicates that the task did not appropriately assess the unit learning outcomes or that the difficulty level of the assessment was too high - see Marking and Feedback Procedure)
- Review of marks to inform future assessment tasks and rubrics
Where the Chief Examiner determines that the above moderation approaches are not appropriate (e.g., for an honours thesis or similar), alternative methods of quality assurance may be required to ensure consistency in marking:
- Using panel marking to minimise subjectivity, especially for oral, performance, practical or laboratory assessment
- Engaging external moderators (i.e. individuals external to the teaching/marking team) to review a sample of marked assessment
- Assigning markers to mark the same question in assignments, texts or examinations composed of multiple sections