Addressing quality challenges in deep learning: The role of MLOps and domain knowledge

Abstract

Deep learning (DL) systems present unique challenges in software engineering, especially concerning quality attributes like correctness and resource efficiency. While DL models excel in specific tasks, engineering DL systems is still essential. The effort, cost, and potential diminishing returns of continual improvements must be carefully evaluated, as software engineers often face the critical decision of when to stop refining a system relative to its quality attributes. This experience paper explores the role of MLOps practices¿such as monitoring and experiment tracking¿in creating transparent and reproducible experimentation environments that enable teams to assess and justify the impact of design decisions on quality attributes. Furthermore, we report on experiences addressing the quality challenges by embedding domain knowledge into the design of a DL model and its integration within a larger system. The findings offer actionable insights into the benefits of domain knowledge and MLOps and the strategic consideration of when to limit further optimizations in DL projects to maximize overall system quality and reliability.

This work is partially supported by the GAISSA project TED2021-130923B-I00, funded by MCIN/AEI/10.13039/501100011033 and the European Union Next Generation EU/PRTR. It is also partially funded by the Joan Oró pre-doctoral support program (BDNS 657443), co-funded by the European Union.

Peer Reviewed

Postprint (author's final draft)