2026-04-14T06:55:50Zhttps://recercat.cat/oai/request
oai:recercat.cat:20.500.14342/60692026-03-19T19:59:01Zcom_2072_482405com_2072_183628col_2072_482408
RECERCAT author Thomas, Llewellyn author Romasanta, Angelo Kenneth author Pujol Priego, Laia 2026-03-19T19:59:01Z 2026-03-19T19:59:01Z 2026-01 0148-2963 https://hdl.handle.net/20.500.14342/6069 https://doi.org/10.1016/j.jbusres.2025.115804 Large Language Models (LLMs) are increasingly viewed as a valuable tool for academic research. While LLMs have some benefits, a ‘crisis of replicability’ in management scholarship mitigates against unrestrained use. In this paper we investigate the reproducibility of LLM analyses. We analyze three LLMs—ChatGPT, Claude and Mistral—over fifteen weeks, testing the consistency, accuracy and their interaction using the same prompts on the same data corpus. While our results demonstrate significant variations in reliability and consistency across the three LLMs, we also show that LLMs can exhibit deterministic and reliable behavior under specific, well-defined constraints. We argue that replicable LLM-based research will rely on understanding and validating the task-specific operational boundaries of the LLM. To ensure the responsible integration of LLMs into management research, we highlight a need for robust frameworks, transparency, ethical guidelines, and ongoing evaluation. We conclude with actionable guidance for management researchers. eng © L'autor/a Attribution-NonCommercial-NoDerivatives 4.0 International Generative AI LLM Replication Reproducibility Consistency Accuracy Jagged competencies: Measuring the reliability of generative AI in academic research info:eu-repo/semantics/article