Exploring AI and assessment – avoid, outrun or embrace
A review of the three main approaches to generative AI in assessment.
The release of ChatGPT in Nov 2022 marked a turning point in the public's perception of artificial intelligence (AI), sparking both excitement and concern in equal measure. In tertiary education, this has led to a heightened focus on how generative AI, especially tools like ChatGPT, is impacting academic integrity and assessment.
Jisc's report from 2021, The future of assessment: five principles, five targets for 2025 and its follow-up, Principles of good assessment and feedback demonstrate a long-standing demand for change in assessment methods within the education sector. These reports highlight the need for evolving assessment practices, and it is hoped that generative AI will act as a catalyst to accelerate these changes.
There are three main approaches to generative AI and assessment - avoid AI, outrun AI or embrace AI - each with specific nuances and considerations. This blog post will discuss and review each of these approaches.
Avoid AI
The ‘avoid’ approach involves minimising the potential use of generative AI tools, such as ChatGPT, in completing assessments. Generative AI technology is becoming increasingly capable of handling a wide range of tasks that were previously thought to be beyond its scope. As generative AI becomes more advanced, this strategy narrows to two key methods:
- A return to traditional supervised exams in controlled environments to prevent the use of AI tools.
- Alternative methods including video submissions, vivas or oral exams.
Benefits: It maintains academic integrity by removing access to generative AI tools and requires little additional work in terms of changing existing assessment formats. Alternative methods such as video submissions can be beneficial in improving communication and presentation skills.
Disadvantages: This approach has many resource and planning challenges for staff and can create stress and anxiety for students/learners. It is also debatable whether traditional assessment methods reflect the skills and culture students/learners will encounter in the workplace.
Outrun AI
The ‘outrun’ strategy focuses on designing assignments and tests that are challenging for generative AI to handle effectively. Initially, this approach leveraged tasks where generative AI struggled, such as interpreting up-to-date information or understanding complex graphs. However, as generative AI tools like ChatGPT continue to evolve rapidly, the range of tasks that AI can't handle is shrinking and currently includes:
- Physical tasks and experiments: Assignments that require physical interaction, such as conducting a laboratory experiment, building a physical model, engaging in fieldwork or a live performance.
- Advanced problem-solving in STEM (Science, Technology, Engineering and Maths): Complex, multi-step STEM problems, especially those requiring the application of advanced theoretical concepts or real-world data interpretation.
Benefits: Staff will be constantly up-to-date with generative AI’s expanding capabilities.
Disadvantages: The constant need to adapt assessments to keep pace with AI advancements will increase staff workload. It will also create additional stress and pressure on staff to stay ahead of developments. Given the lag between assessment creation and completion and the speed of generative AI developments, AI may well have caught up enough to be able to complete the assessment when it comes to taking it.
Embrace AI
The ‘embrace’ approach involves integrating generative AI into the learning and assessment process, recognising its potential to augment education and prepare students/learners for an AI-enabled world. It involves creating authentic assessments to prepare students for an AI enabled workplace, including:
- Allowing the use of AI in an assistive role, for example initial drafting or providing structure, supporting a particular process such as testing code or translating content, giving feedback on content, or proofreading.
- Designing assignments where AI is integrative for example comparing content (AI generated and human generated) researching and seeking answers, playing a Socrative role and engaging in discussion or generating content to be critiqued by students/learners.
Benefits: Embracing AI enhances learning by integrating new technology and preparing students for real-world applications and future employment opportunities.
Disadvantages: It can increase digital inequity if all students do not have equal access to relevant tools. Staff will also need support to redesign assessments.
Creating authentic assessments that are ‘AI proof’ is still a significant challenge. There is no ‘one size fits all’ solution to this problem, and options will be discipline-specific. To support this, Jisc has collaborated with university colleagues to collate ideas on how to move forward.
Published as a set of postcards to stimulate discussions, reflections, and support changes in assessment approaches in an AI-enabled world, this resource is available to all Jisc members across tertiary education. The postcards serve as tools to provoke thought and encourage innovative thinking as educators continue to adapt to the challenges and opportunities presented by generative AI in education.
Conclusion
Navigating the integration of generative AI in educational assessments is a challenge, balancing the need to maintain academic integrity with the technological evolution, and the realities of future work environments. The 'avoid' strategy, with its emphasis on traditional exam environments, offers a controlled, AI-resistant approach, but ignores the AI-enabled workplaces students/learners will be moving to. The 'outrun' method demands continuous adaptation to keep pace with rapid AI advancements.
The 'embrace' approach aligns most closely with preparing students for a future where generative AI is not just a tool, but a fundamental aspect of the workforce, despite its challenges in ensuring equitable access.
Given these considerations, Jisc promotes the 'embrace' approach as the most forward-thinking strategy, particularly for those committed to equipping students/learners with the skills and understanding necessary to succeed in an AI-enabled world. By integrating AI into educational practices, students are not only prepared for future technological landscapes but are also instilled with a critical understanding of how to leverage these advancements responsibly and effectively.
More information
Jisc has recently partnered with the Association of Colleges (AoC) to develop six principles for the ethical use of AI in FE to help colleges and learning providers use AI safely, responsibly and effectively.
Sue Attewell has also developed further guidance on essential resources to enhance your AI journey which can be accessed via Jisc Involve.
To join the conversation around AI in tertiary education sign up to Jisc’s dedicated AI community of practice.
About the author
I co-lead our artificial intelligence activity. Our focus is on supporting our members to responsibly adopt AI. We provide a wide range of thought leadership, practical advice, guidance, and training alongside piloting relevant AI products.