Title:
|
Audiovisual event detection towards scene understanding
|
Author:
|
Canton Ferrer, Cristian; Butko, Taras; Segura, C.; Giró Nieto, Xavier; Nadeu Camprubí, Climent; Hernando Pericás, Francisco Javier; Casas Pla, Josep Ramon
|
Other authors:
|
Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions; Universitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo; Universitat Politècnica de Catalunya. VEU - Grup de Tractament de la Parla |
Abstract:
|
Acoustic events produced in meeting environments may contain useful information for perceptually aware interfaces and multimodal behavior analysis. In this paper, a system to detect and recognize these events from a multimodal perspective is presented combining information from multiple cameras and microphones. First, spectral and temporal features are extracted from a single audio channel and spatial localization is achieved by exploiting cross-correlation among microphone arrays. Second, several video cues obtained from multiperson tracking, motion analysis, face recognition, and object detection provide the visual counterpart of the acoustic events to be detected. A multimodal data fusion at score level is carried out using two approaches: weighted mean average and fuzzy integral. Finally, a multimodal database containing a rich variety of acoustic events has been recorded including manual annotations of the data. A set of metrics allow assessing the performance of the presented algorithms. This dataset is made publicly available for research purposes. |
Abstract:
|
Peer Reviewed |
Subject(s):
|
-Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la parla i del senyal acústic -Àrees temàtiques de la UPC::Informàtica -Human face recognition (Computer science) -Audio signal processing -Face recognition -Motion estimation -Object detection -Sensor fusion -Transforms -Video signal processing -Reconeixement facial (Informàtica) |
Rights:
|
|
Document type:
|
Article - Published version Conference Object |
Published by:
|
Institute of Electrical and Electronics Engineers (IEEE)
|
Share:
|
|