MuSA: A New TEL Platform for Enhancing Self-Reflection and Musical Understanding through Saliency Analysis of Performance Recordings

Abstract

Treball fi de màster de: Master in Sound and Music Computing

Supervisor: Rafael Ramirez-Melendez

Co-Supervisor: Suvi Haeaerae

This thesis explores how a technology-enhanced learning (TEL) tool can improve music practice by addressing a critical, often-neglected component of skill development: the reflection phase. It focuses on the development and evaluation of MuSA (Musical Salience Analyzer), an application designed to provide a pedagogicallygrounded platform for analyzing recorded performances to make reflection more efficient and effective. MuSA’s design is informed by key educational theories, including the Talent-Development-in-Achievement-Domains (TAD) Music Model and learner-centered teaching (LCT) principles like scaffolding, self-regulated learning (SRL), and self-directed learning (SDL). Its central feature is saliency analysis, which algorithmically identifies key moments in a performance based on variability in musical features such as pitch, dynamics, and tempo. Unlike tools that offer prescriptive, "correct/incorrect" feedback, MuSA encourages a learner’s own interpretation. As an accessible, web-based platform, it allows users to upload or record audio for analysis independently of a teacher. To evaluate MuSA’s effectiveness, a mixed-methods, within-subjects study was conducted with 14 participants. While the study’s small sample size limited statistical power, the findings pointed to several exploratory trends. The data suggested a differential impact based on musical feature and experience level, with dynamics showing the most consistent trend toward objective and perceived improvement. The analysis also suggested a potential expertise reversal effect, where trends showed intermediate musicians gaining from the feedback while advanced musicians experienced neutral or slightly negative changes. Furthermore, the study’s self-awareness metrics indicated a general misalignment between participants’ self-ratings and objective performance, highlighting a core challenge in the self-reflection phase of independent practice. In conclusion, MuSA offers a potential contribution to TEL for music by leveraging computational analysis to provide targeted insights that can scaffold the reflective process. Although the quantitative results were inconclusive, positive qualitative feedback validates the demand for such a tool. This work provides a functional prototype and a research infrastructure for collecting labeled recording data, demonstrating the dual role ofMuSA as both a learning support system and a potential research tool.