Overview of DETESTS-Dis at IberLEF 2024: DETEction and classification of racial STereotypes in Spanish - Learning with Disagreement

Abstract

This paper presents an overview of the DETESTS-Dis shared task as part of the IberLEF 2024 Workshop on Iberian Languages Evaluation Forum, within the framework of the SEPLN 2024 conference. We proposed two hierarchical subtasks: In subtask 1, participants had to determine the presence of stereotypes in the texts. For subtask 2, participants had to decide which texts labeled with stereotypes were implicit stereotypes. The DETESTS-Dis dataset contains 12,111 comment sentences and tweets in response to newspaper articles and verified racial hoaxes involving immigration in Spanish. 15 teams signed up to participate, 6 of which sent runs, and 3 of them sent their working notes. In this paper, we provide information about the training and test datasets, the systems used by the participants, the evaluation metrics of the systems and their results.