An assessment activity was created to put skills in place to critique output from ChatGPT effectively and work with the Large Language Model (LLM) through peer feedback. ChatGPT was used in the assessment with the aim of empowering students with literacy in GenAI and encouraging their engagement and comprehension of this technology.
The inspiration for this activity stemmed from the anticipation of students entering the school year equipped with newfound proficiency in Generative Artificial Intelligence (GenAI), particularly in utilising ChatGPT for assessments. Recognising the transformative potential of GenAI literacy, we proactively designed a structured approach to facilitate students' engagement and comprehension of this emerging technology. We developed a series of four scaffolded tasks embedded within a writing portfolio, strategically crafted to encourage students to not only refine their writing skills but also deepen their understanding of GenAI. Our primary goal was to provide a platform for students to explore and critically assess the capabilities and limitations of GenAI early on, without imposing rigid expectations.
 | Facilitate an interactive tutorial tailored to specific unit content. |
 | Following the tutorial, students were presented with GenAI-generated outputs, prepared by educators, relevant to the tutorial topics. Working in groups, students meticulously analysed these outputs, dissecting them sentence by sentence to assess accuracy, clarity, and potential biases. Subsequently, students collaboratively sourced accurate responses to the provided statements, leveraging their collective insights and knowledge. Group members offered feedback to their peers using FeedbackFruits. Students returned to FeedbackFruits to upload feedback for review by peers. |
 | In the second task, groups were tasked with critiquing additional ChatGPT outputs, crafting a conversational-style essay spanning 600 words in response. Here, students had the flexibility to integrate as much or as little of the output as deemed necessary. The skills honed in the initial task served as a scaffold for this more advanced activity, empowering students to engage critically and constructively with GenAI-generated content. |
 | Transitioning to the third activity, students embarked on an individual writing task, drawing upon the skills and insights cultivated in prior sessions. This task required students to apply their acquired knowledge and analytical abilities to produce accurate outputs independently, thereby reinforcing their understanding and proficiency in navigating GenAI-generated content autonomously. |
The outcome of implementing structured activities involving GenAI was highly positive. Student engagement levels were notably high, driven by a sense of competition to provide the most accurate responses by leveraging various resources. This approach not only facilitated deeper engagement with GenAI but also helped set clear expectations at the beginning of the semester.
By guiding students through structured tasks, we were able to effectively communicate expectations and establish frameworks for independent work with GenAI. Moreover, the group-oriented nature of the activities encouraged collaboration among students, fostering teamwork and cohort relationships early in the semester. Overall, the outcome underscored the value of integrating GenAI literacy into the curriculum, laying a solid foundation for students to navigate and utilise AI technologies effectively throughout their academic journey.
This exemplar requires a medium level of effort to implement.
Recommended resources and training:
Identify a tutorial or lecture where a post-class activity would be beneficial for reinforcing learning objectives. This could involve choosing a session with ample resources or a topic that could benefit from deeper exploration. Develop a prompt that is sufficiently broad to elicit an interesting response from the AI but also contains elements that students can critique for accuracy and bias. Decide whether students will work individually or in groups. Group work can facilitate teamwork and cohort building, especially if students are meeting for the first time.
Establish clear deadlines for completing the activity. Consider providing time immediately after the tutorial for group collaboration and then setting a deadline for submission. Provide guidance on how feedback should be given. Encourage constructive criticism rather than mere praise.
Rather than imposing bans on AI usage, consider introducing students to ChatGPT in a structured, transparent manner. Provide guidelines on how to engage with GenAI tools, empowering students to use them effectively while minimising concerns about academic integrity.
Focus on the learning benefits as part of the learning process. By framing GenAI integration as a tool to enhance understanding and critical thinking skills, students may be more inclined to engage positively with the technology.
Foster a collaborative learning environment where students can discuss and critique each other's work, including GenAI content. This not only promotes deeper understanding but also helps students develop essential skills in giving and receiving feedback. Establish a clear feedback philosophy early on in the course. Emphasise the importance of constructive feedback and open communication between students and instructors.
Take the opportunity to discuss expectations for the course and for using AI tools. Clarify how GenAI tools will be integrated into assessments and how they should be utilised to support learning objectives.
Here are some additional resources that you can browse to help you implement this assessment.
