Towards trustworthy AI research assistants: leveraging knowledge graphs for knowledge synthesis

Abstract

Treball fi de màster de: Erasmus Mundus joint Master in Artificial Intelligence (EMAI)

Supervisors: Dr. Alessandro Zani, Prof. Dr. Vicenç Gómez

The use of LLMs is becoming more widespread every day. Its use of purpose varies from answering questions, evaluating information, and summarizing a large amount of information. This capability of LLMs also helps researchers who need to read and follow a lot of papers. Making sense of scientific research is more important than ever for informed decisionmaking. It is becoming nearly impossible for researchers to keep up and piece everything together manually with the overwhelming number of new studies being published every day. AI-powered research assistants, especially those built on large language models are beginning to help to find, analyze, and summarize huge volumes of information. However, how can we be sure that these summaries are accurate and reliable? Recent studies point to a promising solution, the grounding of LLMs with structured formats such as knowledge graphs (KGs), which can help improve both the reliability, explainability, and quality of the information they produce. This thesis explores how to generate knowledge graphs from unstructured text, with a focus on understanding and comparing different methods and configurations. The main goal is to evaluate how well these approaches capture useful information, both in terms of quantity and quality, by using a recently published benchmark as a point of reference. Ultimately, the findings are designed to support the integration of knowledge graphs into LLM-based research assistants, helping to make knowledge synthesis more accurate, reliable, and effective.