The development and exploratory analysis of the Writing and AI Knowledge Scale (WAIKS)

Abstract

Comunicació presentada al EuroCALL 2025 - Advancing CALL: new research agendas, celebrada a Milà(Itàlia) del 27 al 30 d'agost de 2025.

Generative AI chatbots powered by large-language models (e.g., ChatGPT, Gemini, CoPilot) offer both opportunities and challenges for L2 writers at university. Their ease of use and the apparent fluency of the texts produced make them popular with students. However, this convenience may limit opportunities for deeper processing and learning. Furthermore, AI-generated texts may contain contextual, stylistic, and pragmatic errors; plagiarised content; and implicit biases related to gender, race, and other issues which novice writers struggle to detect. The Writing and AI Knowledge Scale (WAIKS) questionnaire is envisaged as a way for teachers to quickly assess students¿ attitudes towards and knowledge of these issues. This study evaluates a WAIKS prototype with 128 L2 English writers (CEFR B2-C2) at a Spanish university. In an exploratory analysis of the prototype, Principal Components Analysis identified three constructs -Academic Appropriateness, Impression, and Intention to Use. Comparisons across language proficiency and academic experience groups suggest that students with higher English proficiency may be more aware of chatbot limitations, while students with more university experience may have greater practical motivation to use chatbots. These tentative results support the construct validity of WAIKS. Further evaluation is needed with a broader range of participants. This will help establish benchmarks for interpreting scores.

The development and evaluation of WAIKS was supported by Universitat Pompeu Fabra via the PlaCLIK programme (PlaCLIK project: E2024018266).