To access the full text documents, please follow this link:

Audiovisual event detection towards scene understanding
Canton Ferrer, Cristian; Butko, Taras; Segura, C.; Giró Nieto, Xavier; Nadeu Camprubí, Climent; Hernando Pericás, Francisco Javier; Casas Pla, Josep Ramon
Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions; Universitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo; Universitat Politècnica de Catalunya. VEU - Grup de Tractament de la Parla
Acoustic events produced in meeting environments may contain useful information for perceptually aware interfaces and multimodal behavior analysis. In this paper, a system to detect and recognize these events from a multimodal perspective is presented combining information from multiple cameras and microphones. First, spectral and temporal features are extracted from a single audio channel and spatial localization is achieved by exploiting cross-correlation among microphone arrays. Second, several video cues obtained from multiperson tracking, motion analysis, face recognition, and object detection provide the visual counterpart of the acoustic events to be detected. A multimodal data fusion at score level is carried out using two approaches: weighted mean average and fuzzy integral. Finally, a multimodal database containing a rich variety of acoustic events has been recorded including manual annotations of the data. A set of metrics allow assessing the performance of the presented algorithms. This dataset is made publicly available for research purposes.
Peer Reviewed
Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la parla i del senyal acústic
Àrees temàtiques de la UPC::Informàtica
Human face recognition (Computer science)
Audio signal processing
Face recognition
Motion estimation
Object detection
Sensor fusion
Video signal processing
Reconeixement facial (Informàtica)
Attribution-NonCommercial-NoDerivs 3.0 Spain
Institute of Electrical and Electronics Engineers (IEEE)

Show full item record

Related documents

Other documents of the same author

Butko, Taras; Canton Ferrer, Cristian; Segura Perales, Carlos; Giró Nieto, Xavier; Nadeu Camprubí, Climent; Hernando Pericás, Francisco Javier; Casas Pla, Josep Ramon
Butko, Taras; Temko, Andrey A.; Nadeu Camprubí, Climent; Canton Ferrer, Cristian
Butko, Taras; Gonzalez Pla, Fran; Segura Perales, Carlos; Nadeu Camprubí, Climent; Hernando Pericás, Francisco Javier
Canton Ferrer, Cristian; Segura Perales, Carlos; Casas Pla, Josep Ramon; Pardàs Feliu, Montse; Hernando Pericás, Francisco Javier