Human Intervention and AI in Higher Education English Test Designing: Benefits, Challenges, and Recommendations

The rapid advancements in generative artificial intelligence (genAI) have produced significant research in recent years, highlighting the advantages of using genAI in language test designing. These advantages encompass time saving, the generation of test papers, and the enhancement of students’ experiences during stressful assessments. This paper seeks to underline the importance of human intervention in English test preparation by analysing various genAI outcomes informed by direct experience and observation. Specifically, this study investigates the benefits and challenges associated with developing genAI-based English assessment tests for 160 Italian undergraduate students at an upper-intermediate level of English proficiency, using ChatGPT 3.5. I argue that human intervention, particularly through well-crafted prompts, is essential for achieving effective results. This paper advocates for a thoughtful approach to integrating AI in English test designing. The objective is to equip language teachers and practitioners with vital recommendations for test preparation. This guidance will assist them in navigating the intersection of AI and test design, fostering a nuanced understanding of the benefits and challenges posed by AI technology. It also emphasises the necessity of balancing technological reliance with the integrity of higher-order thinking skills. In sum, the paper stresses that human supervision and intervention are critical for producing reliable outcomes, as failures in genAI-based tests are not uncommon. While the primary focus of this study is on the context of Italian undergraduates, it also explores broader implications for the use of AI in language education globally, reinforcing the universal need for human intervention to maintain quality and ethical standards.