2026-04-13T01:17:43Zhttps://recercat.cat/oai/requestoai:recercat.cat:11351/128202025-10-24T10:20:58Zcom_2072_378070com_2072_378040col_2072_378092
RECERCAT
author
Lang, Siegmund
author
Vitale, Jacopo Antonino
author
Galbusera, Fabio
author
Fekete, Tamás F.
author
Charles, Yann Philippe
author
Haddad, Sleiman
author
boissiere, louis
author
Núñez-Pereira, Susana
2025-10-24T10:20:58Z
2025-10-24T10:20:58Z
2025-03-21T10:44:30Z2025-03-21T10:44:30Z20242025-03
http://hdl.handle.net/11351/12820
Adolescent idiopathic scoliosis; Large language models; Patient educationEscoliosi idiopàtica de l'adolescent; Models de llenguatge extens; Educació pel pacientEscoliosis idiopática del adolescente; Modelos de lenguaje extenso; Educación para el pacientePurpose
Large language models (LLM) have the potential to bridge knowledge gaps in patient education and enrich patient-surgeon interactions. This study evaluated three chatbots for delivering empathetic and precise adolescent idiopathic scoliosis (AIS) related information and management advice. Specifically, we assessed the accuracy, clarity, and relevance of the information provided, aiming to determine the effectiveness of LLMs in addressing common patient queries and enhancing their understanding of AIS.
Methods
We sourced 20 webpages for the top frequently asked questions (FAQs) about AIS and formulated 10 critical questions based on them. Three advanced LLMs—ChatGPT 3.5, ChatGPT 4.0, and Google Bard—were selected to answer these questions, with responses limited to 200 words. The LLMs’ responses were evaluated by a blinded group of experienced deformity surgeons (members of the European Spine Study Group) from seven European spine centers. A pre-established 4-level rating system from excellent to unsatisfactory was used with a further rating for clarity, comprehensiveness, and empathy on the 5-point Likert scale. If not rated 'excellent', the raters were asked to report the reasons for their decision for each question. Lastly, raters were asked for their opinion towards AI in healthcare in general in six questions.
Results
The responses among all LLMs were ‘excellent’ in 26% of responses, with ChatGPT-4.0 leading (39%), followed by Bard (17%). ChatGPT-4.0 was rated superior to Bard and ChatGPT 3.5 (p = 0.003). Discrepancies among raters were significant (p < 0.0001), questioning inter-rater reliability. No substantial differences were noted in answer distribution by question (p = 0.43). The answers on diagnosis (Q2) and causes (Q4) of AIS were top-rated. The most dissatisfaction was seen in the answers regarding definitions (Q1) and long-term results (Q7). Exhaustiveness, clarity, empathy, and length of the answers were positively rated (> 3.0 on 5.0) and did not demonstrate any differences among LLMs. However, GPT-3.5 struggled with language suitability and empathy, while Bard’s responses were overly detailed and less empathetic. Overall, raters found that 9% of answers were off-topic and 22% contained clear mistakes.
Conclusion
Our study offers crucial insights into the strengths and weaknesses of current LLMs in AIS patient and parent education, highlighting the promise of advancements like ChatGPT-4.o and Gemini alongside the need for continuous improvement in empathy, contextual understanding, and language appropriateness.Open Access funding enabled and organized by Projekt DEAL.
Attribution 4.0 International http://creativecommons.org/licenses/by/4.0/ info:eu-repo/semantics/openAccess
Intel·ligència artificial
Metge i pacient
Escoliosi
Educació dels pacients
Adolescents
HEALTH CARE::Health Care Facilities, Manpower, and Services::Health Services::Preventive Health Services::Health Education::Patient Education as Topic
PHENOMENA AND PROCESSES::Mathematical Concepts::Algorithms::Artificial Intelligence
DISEASES::Musculoskeletal Diseases::Bone Diseases::Spinal Diseases::Spinal Curvatures::Scoliosis
PSYCHIATRY AND PSYCHOLOGY::Behavior and Behavior Mechanisms::Psychology, Social::Interpersonal Relations::Professional-Patient Relations::Physician-Patient Relations
NAMED GROUPS::Persons::Age Groups::Adolescent
ATENCIÓN DE SALUD::instalaciones, servicios y personal de asistencia sanitaria::servicios de salud::servicios preventivos de salud::educación en salud::educación de pacientes como asunto
FENÓMENOS Y PROCESOS::conceptos matemáticos::algoritmos::inteligencia artificial
ENFERMEDADES::enfermedades musculoesqueléticas::enfermedades óseas::enfermedades de la columna vertebral::desviaciones de la columna vertebral::escoliosis
PSIQUIATRÍA Y PSICOLOGÍA::conducta y mecanismos de la conducta::psicología social::relaciones interpersonales::relaciones profesional-paciente::relaciones médico-paciente
DENOMINACIONES DE GRUPOS::personas::Grupos de Edad::adolescente
Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion