Competency-based peer assessment with FeedbackFruits

In 2019, we transformed the unit with help from the Monash Learning Transformation Team. While we implemented the new unit in 2020, cost became prohibitive and the unit was paused in 2021. We then looked for a way that we could gain similar educational outcomes for students i.e. they learn core science communication skills but still reduce the cost of delivering the unit.

In 2022, I re-transformed the unit from:

Switching from grade to competency based assessment was quite a learning curve for me. All of my prior assessment design has been grade based, so my first step was to reach out to colleagues who had designed competency based assessment and consult the literature.

After getting my head around how competency based assessment worked, I also brainstormed what science communication competencies would be relevant for first year students, relative to their assessments.

This led to me developing two assessments, with a third student-generated component:

Assessment regime design

Students produced these with the aim to meet the 12 defined science communication competencies. They were then required to self-review their own popular media article and infographic, before providing a peer review of two popular media articles and two infographics from other students (AT3).

In conducting the self and peer review for AT1 and AT2, students aim to meet the six self/peer review competencies. Students must meet all 18 competencies to pass the unit.

If a student has not met the 18 competencies to pass the unit they are offered additional assessment to be completed by the end of week 12.

I designed this so that students would be able to use feedback comments to edit their initial assessments (AT1 and AT2) and submit these as a portfolio in the aim to meet any remaining science communication competencies.

If they are missing self/peer review competencies they need to complete these based on popular media articles or infographics chosen by me.

What I have learnt from the first implementation are the following:

• Students were very confused about what competency based assessment was, even after much explanation
• Students did not take competency based assessment seriously, thinking it was much easier than grade based assessment until after their initial results were returned.
  • After AT1 – AT3 only 130 of 475 students had passed all competencies and the unit.
  • After the additional assessment another 191 students met the remainder of their competencies and passed the unit.
  • Overall, 321 out of 475 students (i.e. 67%) achieved a pass in S1 2022.
  • Of the 154 students that failed, 96 are missing two or less of the competencies (near fail) and will be eligible for unit-based additional assessment.
• There is a direct relationship between students who completed the weekly modules (i.e. geared toward teaching them assessment skills) and those who passed the unit.
• In week 12, I offered 16 hours of drop-in sessions. Students could come along and ask questions and get help from me to meet their remaining competencies. Those that came along to these sessions had greater success meeting their competencies over those who did not.
• As this is the first iteration there is little to no feedback from students in relation to the unit. The only feedback I have received via email was that students were really happy with the amount of effort I put into answering all of their queries via the forums.

Students’ work

Once you have decided on your assessment regime and obtain approvals from your Faculty, you would need to set up both Moodle Gradebook and Feedback Fruits to enable grading and marking.

Set up the Moodle gradebook for competency based assessment

Setting up the Moodle gradebook for competency based assessment is more complex than then the standard Moodle gradebook setup but getting it right early will save you a lot of time and effort in the long run. Below is an example of how I set up the Moodle gradebook in SCI1000 for competency-based assessment.

Click on the plus icons below to learn more about how the SCI1000 Gradebook was set up for competency-based assessment. You can click the icon in the top right corner to view in fullscreen. Click esc at any time to exit full screen.

Set up the FeedbackFruits Peer review activity

The FeedbackFruits Peer review activity was simple and quick to set up. Instructions to add a FeedbackFruits activity to your Moodle unit can be found here. You can read about how to set up the Peer Review activity here.

Expand the accordion below to see the steps for setting up the Peer Review activity for SCI1000

• Emphasise on every occasion how important the online weekly modules are to their success in the unit.
  Even though I did this in S1, I am going to pull out the data and actually present this to students in our first seminar in S2. Seeing graphical data of pass rates vs completion of online modules may have more impact than simply telling them.

• Think very carefully about the competencies you want students to achieve and map these across both assessments.
  This way, they get two chances to meet the competencies before additional assessment is required.

• Have enough staff on board to check all student’s competences and their peer review skills for both assessments.
  In S1 I was the lone staff member to 475 students. While I quality checked 50% of students in A1 and 50% in A2 for the science communication competencies, I was unable to check that they had done their peer reviews to the standard required. This will be remedied in the next iteration.

• Make sure the Gradebook is set up correctly for competency based assessment.
  Getting this right will save you a lot of time behind the scenes. I was lucky to get advice from Monash Education Academy on how to do this. In S1 I thought I had it right, but part way through realised it needs a bit more tweaking to minimise processing outside the gradebook.

• Offer drop-in sessions at key points throughout the semester
  For example you can offer drop-in sessions before assessment submission, after they receive results for each semester and prior to submission of additional assessment. I was developing and delivering the unit at the same time (not recommended by the way) and could only offer drop-in sessions in week 12. But these were very well received by students.