Explaining AI Applications through Workflow Provenance

Abstract

Understanding the decision-making process of AI systems is a necessary step for building trust in the results they produce. Among the various approaches that address explainability in AI, this thesis focuses on how workflow provenance, the automatic record of steps and data transformations during model development and execution, can provide insight into the behavior of models. The main contribution of this work is the extension of COMPSs, a distributed workflow management system, to support very detailed provenance metadata registration. We store the metadata using the RO-Crate format, which ensures interoperability and reproducibility of experiments. To demonstrate the usefulness of the captured provenance metadata, we present it through knowledge graph visualizations, that enable users to explore, filter, and validate experiments interactively. By doing so, this thesis contributes not only to the explainability of COMPSs applications but also to broader efforts for achieving trustworthy and transparent AI.