Schlagwort: Hochschulen

Text: Mònica Feixas

In the actual context of higher education, the emergence of artificial intelligence (AI) tools introduces a multitude of challenges to conventional assessment practices. While we are witnessing that certain teaching and learning activities may eventually be performed by AI to a relatively good standard, we cannot fully rely on the generated content because it can exhibit deficiencies in accuracy, comprehending context, coping with intricate analysis or being ethical. For rigorous scholarly pursuits, it is necessary to empower students to critically discern and interpret data, to identify bias and ensure fairness, and to make choices aligned with human values, that is: to form evaluative judgements (EJ). This blog is about practices that foster EJ within the context of authentic assessment tasks, answering questions such as: What precisely constitutes an authentic assessment task? What practices promote EJ in the context of authentic assessment tasks? How can we effectively leverage tools like ChatGPT to facilitate the development of evaluative judgement?

Authentic Assessment Tasks in Higher Education: a Response to traditional Assessments

Authentic assessment tasks have emerged as a meaningful response to address the limitations of conventional assessment methods in higher education (Clegg & Bryan 2006). They require students to perform as professionals within the actual social and physical contexts of a specific field, demanding the demonstration of skills and knowledge reflective of real-world scenarios.  

Authentic assessment aims to replicate the tasks and performance standards typically found in the world of work and has been found to have a positive impact on student learning, autonomy, motivation, self-regulation and metacognition; abilities highly related to employability.

(Villarroel et al. 2018)

Different displays of knowledge and performance can be encouraged based on the selected assessment formats employed to assess students‘ learning:

While traditional assessment formats focus on the demonstration of Knowing (e.g. through factual tests or MCQs) or Knowing How (context-based tests, MCQs, problem solving), authentic assessment also emphasize Showing How (performance based, objective school observations, problem-based learning, scenarios, portfolios) and Showing Doing (performance based tasks, work/professional experience, patient care, pupils’ teaching) (Sambell & McDowell 1998). 

Designing authentic tasks can be done by following the five-dimensional framework from Gulikers, Bastiaens and Kirschner (2004) (reproduced in Figure 2) with pertinent questions in relation to each dimension. Another alternativ is Sambell’s guide. In both cases, effective assessments work in tandem with teaching and learning activities to help students develop long-term approaches to learning.

Examples of authentic assessment tasks can be found in Jon Mueller’s Authentic Assessment Toolbox, or Kay Sambell and Sally Brown.

Evaluative Judgement: a crucial Skill in the Age of AI 

Following Villarroel et al. (2018), authentic assessments should include at least three components: 

• Realism, to engage students with problems or important questions relevant to everyday life; 
• Cognitively challenging tasks that prompt students to develop and use higher levels of thinking to use knowledge, process information, make connections and rebuild information to complete a task (rather than low-level recall or reproduction of facts); 
• Opportunities to develop Evaluative Judgement and enhance the self-regulation of their own learning. 

Evaluative judgement is the capability to make decisions about the quality of work of oneself and others.

Tai et al. (2018)

In the age of AI, supporting students navigate the fake news world, veracity of data and reflect the work being presented by ChatGPT or other artificial intelligence tools is crucial. It is only possible if students develop a deep understanding of topics and of quality and help them assess quality of a product or performance. EJ is all about engaging with grading criteria, improving the capacity to appreciate the features of «quality» or excellence in complex outputs and developing the ability to provide, seek and act upon feedback. 

EJ is additionally a skill that interacts with self-regulated learning. When students develop an understanding of quality, they are better able to apply feedback, and become less reliant on external sources of feedback. Students who can self-regulate and judge their own work can be more autonomous in their learning. It is suggested that students with these attributes may make a smoother transition into the workforce (Tai et al. 2018). 

EJ is also relevant from a perspective of inclusive education. A Delphi Study with 10 international experts on authentic and inclusive assessment showed that activities which develop EJ include discussions of quality with their students, listen to students‘ perspectives and have the potential to be more inclusive by ensuring that all students have a shared understanding of standards and criteria (Feixas & Zimmermann 2023).

Practices supporting the Development of Students’ evaluative Judgement

In order to apply EJ we have to consider its two key components: the contextual understanding of the quality of work, and the judgement (and articulation thereof) of specific instances of work. This can be applied to the work or performance of both self and others.   

Research suggest the following practices to developing the EJ components (Bouwer et al. 2018; Tai et al. 2018; Sridharan & Boud 2019):

1. Exemplars:  

• Students receive various examples of expected standards for evaluating their own and others‘ performance, including progress notes, reports, learning goals, reflection sheets, and intervention strategies. They are provided alongside grading criteria, and can be used for  review, or students can discuss exemplar assignments in groups. 
• Video-feedback, like «live marking» screencasts, can also be utilized to demonstrate different levels of work quality. See the three examples by Dr. Nigel Frances (University of Swansea):  

• Poor Report (Youtube)
• Average Report (Youtube)
• Good Report (Youtube)

2. Assessment criteria and rubrics: 

• Checklists, templates, or rubrics are provided to help students reflect on their achievement of competencies. 
• Students engage with criteria by discussing the meaning and distinguishing features of work at each level of the rubric. 
• Involving students in translating generic grade descriptors into assignment-specific grading criteria, and involving students in designing own rubrics with ChatGPT enhances their understanding. 

3. Peer-review and feedback: 

• Students engage in providing feedback on their peers‘ work-in-progress based on the grading criteria, focusing on elements where they can offer valuable insights, such as i.e. argument strength in the case of a ChatGPT-text. 

4. Self-appraisal or self-assessment:  

• Students appraise their own work against grading criteria to show their development of EJ.  
• They submit cover sheets where they self-assess their work before assessment and receive feedback about it. Such feedforward supports improvement from one task to another (Sadler, Reimann & Sambell 2022).   
• This higher-level thinking process involves reflection, identifying potential improvements, and working towards integration in an ongoing manner. 

A fictitious Example of an Authentic Assessment Task in an Educational Psychology Course and the Use of ChatGPT to enhance Evaluative Judgement Practices

Task-Title: Inclusive Education Initiatives – An Educational Psychology Project for Social Impact 

In this group task, students design an inclusive education initiative that not only addresses educational psychology principles but also seeks to foster diversity, equity, and inclusion in educational settings. The goal is to create a project that promotes an inclusive learning environment, where every pupil thrives academically and socially. Rubrics, self-and peer-assessment options, exemplars, and feedback are practices deployed to strengthen their evaluative judgement skills.  

Part 1: Identifying the Need

Students conduct research on challenges in marginalized communities‘ access to quality education. They are allowed to use AI-tools after their initial research to broaden their understanding of the educational psychology concepts related to the identified communities. AI can assist, for example, in providing new data, visualisations, translating information from other languages, condensing content summaries or responding questions. Students‘ critical analysis and ethical review of the generated content is crucial in interpreting results and making summaries or recommendations. The teacher afterwards provides feedback to the groups to improve the needs assessment. 

Part 2: Project Design

Based on their research, students develop evidence-based inclusive education initiatives. ChatGPT can be utilized to help students with intervention strategies, best practices, and approaches used in similar contexts. A possible prompt is: «Identify evidence-based interventions that have demonstrated success in narrowing the education gap among underserved populations.» Peer-assessment is encouraged to critically evaluate their different project designs and identify differences between versions of ChatGPT. Groups include improvements done after peer-feedback. 

Part 3: Social Impact Assessment

Students describe the potential social impact of their initiatives and self-evaluate it with a rubric containing criteria about how the project can contribute to breaking down barriers, promoting social cohesion, and enhancing educational opportunities for the targeted group. 

Part 4: Stakeholder Engagement

Students present their projects to stakeholders and utilize feedback to enhance their engagement strategies, ensuring buy-in and long-term sustainability. After the presentation and before the final submission, teacher offers exemplars based on successful projects in similar contexts. 

Final Reflection:

Students utilize a rubric to self-assess their projects, reflecting on strengths, limitations, and ethical considerations. The final project is submitted alongside an individual reflection highlighting aspects of the use of EJ. Final feedback is provided by the teacher to foster growth and improvement.

INFOBOX

Empowering academics to promote Evaluative Judgement – three training opportunities:

We want to empower teachers to design authentic assessment practices to better create work-ready graduates, who are able to operate independently in rapidly evolving, technologically-enabled environments and to promote EJ as means to judge the quality of work through standards.
At ZHE, we offer a half-day course online on Authentic Assessment and Evaluative Judgement: «How to judge the quality of my own work and that of others, including ChatGPT? Evaluative Judgement in the context of authentic assessment tasks». 27th September 2023, 13–17h, online.

From February to May 2024, the PH Zürich offers a one-time course: «Beyond Exams: Designing Authentic Assessment and Feedback Practices». The 1.5 ECTS module takes place in the context of the international project «PEER-Net» of the Department Projects in Education (PH Zürich). The course is taught in a team-teaching format by experts in assessment and feedback of the Faculty of Education of the University of Pristina in Kosovo and the ZHE. The participation in this module includes a reciprocal visit (Swiss scholars to Kosovo and the colleagues of Kosovo to Zurich), classroom visits and discussion with students.

In the context of our CAS Hochschuldidaktik, the module «Assessment und Evaluation» offers a comprehensive exploration of these subjects. 26th October to 7th December, PH Zürich. Enrol now!

About the Author

Mònica Feixas is a lecturer at the Centre for Teaching and Learning in Higher Education (ZHE) at the Zurich University of Teacher Education.