GenAI: Evaluate

Incorporating GenAI into learning and assessment is novel and should be evaluated. We can gather insights to enhance and innovate future assessments by evaluating and reflecting on assessment outcomes, the task design and instructions provided, and the learning opportunities given to students regarding the ethical and responsible use of GenAI and the questions they asked.

Key aspects to consider in the evaluation of GenAI use in assessment tasks and associated learning activities

1. GenAI breaches

Were any GenAI-related breaches identified? Identifying and understanding such instances will help us refine the implementation of GenAI in future assessments, identify areas for educational opportunities for students, and mitigate potential risks.

2. Grade distribution

Analyse the grade distribution of students - how well did students perform? This data can reveal whether the integration of GenAI positively influenced learning outcomes and whether any disparities or challenges arose among different student groups.

3. Student feedback

Collect feedback from students (formally or informally). What were their experiences, attitudes, and perceptions on using GenAI in the learning process? Positive feedback can highlight successful aspects, while negative feedback can pinpoint areas for improvement.

4. Your own reflections

As an educator, reflect on aspects such as student engagement, areas of confusion and the effectiveness of the instructions.

Consider the following questions:

  • Were students generally receptive to the use of GenAI in their learning?
  • Did students encounter specific challenges or confusion related to the ethical and responsible use of GenAI?
  • Were there any particular questions or concerns raised by students regarding the task?
  • How did students' work compare to the standard expected in non-AI-assisted assessments?
  • Did the integration of GenAI lead to any notable improvements or challenges in the learning experience?
  • What were the challenges for you as an educator in ensuring acceptable limits were not breached in student work?

Use your evaluation and reflection to inform Design for the next iteration of the assessment.

Once you have considered the Evaluate aspects of your assessment, continue with the rest of the assessment design process by clicking on Design, Analyse, Act, Inform, Educate and Check.   


Note: Given the rapidly evolving nature of GenAI technologies and largely opinion-based and low-level evidence on emerging practices for use in higher education, this resource represents the status quo at the time of writing (Aug 2023). As changes to policies and technology develop and evidence for best practice emerges, practice recommendations as outlined here are likely to continue to change and develop.