2026-04-14T04:41:38Zhttps://recercat.cat/oai/request

oai:recercat.cat:2117/4418682026-01-21T09:29:29Zcom_2072_1033col_2072_452950

Multimodal sensing prototype for robust autonomous driving under adverse weather conditions Mas Giménez, Gerard de Subirana Pérez, Adrià Garcia Gómez, Pablo Bernal Pérez, Eduard Casas Pla, Josep Ramon Royo Royo, Santiago Universitat Politècnica de Catalunya. Doctorat en Enginyeria Òptica Universitat Politècnica de Catalunya. Centre de Desenvolupament de Sensors, Instrumentació i Sistemes Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions Universitat Politècnica de Catalunya. Departament d'Òptica i Optometria Universitat Politècnica de Catalunya. GREO - Grup de Recerca en Enginyeria Òptica Universitat Politècnica de Catalunya. IDEAI-UPC - Intelligent Data sciEnce and Artificial Intelligence Research Group Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Radiocomunicació i exploració electromagnètica Adverse weather Autonomous driving Calibration Cameras Imaging systems LIDAR Prototyping Current autonomous driving datasets face significant limitations in adverse weather conditions diversity and sensor generalization. Additionally, commonly used sensors like visible cameras or rotating LiDARs, struggle to perform under harsh weather conditions. To address these challenges, this work introduces a multimodal data acquisition system that integrates high-resolution solid-state LiDAR, automotive RADARs, a combination of visible, thermal, SWIR, and polarimetric cameras, and a GNSS/INS system for odometry and localization. This diverse sensor suite ensures robust performance in low-visibility environments, such as fog or heavy rain, by providing complementary and redundant information. Controlled by an autonomous-safe Nvidia DRIVE AGX with a ROS-based architecture, the system enables precise spatial calibration, temporal synchronization, and realtime data fusion and perception algorithms across an overlapped field of view of 60°x 20°. With this system, this work aims to publish in the coming months an open-source multimodal labeled dataset for autonomous driving complemented by synthetic data generation through a digital twin in Nvidia’s Omniverse platform. The data set will have a similar structure as the well-known nuScenes, and will have a similar developer kit to navigate through it with ease. The data set will support a wide range of autonomous driving applications like 3D multimodal object detection and tracking, SLAM, depth completion, and perception enhancement through challenging scenarios. This work has been sponsored by the Spanish Ministry of Science through projects TED2021-132338B-I00 and PID2023-152427OB-I00 and by the European Commission through project GA-101139941. This work is part of a Ph.D. thesis with grant number 2023 FI-1 00229 co-funded by the European Union. Postprint (author's final draft) 2025 Conference lecture De Mas, G. [et al.]. Multimodal sensing prototype for robust autonomous driving under adverse weather conditions. A: SPIE Optical Metrology. «Optical Measurement Systems for Industrial Inspection XIV: 23-27 June 2025, Munich, Germany». Washington: International Society for Photo-Optical Instrumentation Engineers (SPIE), 2025, ISBN 9781510690431. DOI 10.1117/12.3062388 . 9781510690431 https://hdl.handle.net/2117/441868 10.1117/12.3062388 eng https://www.spiedigitallibrary.org/conference-proceedings-of-spie/13567/135672E/Multimodal-sensing-prototype-for-robust-autonomous-driving-under-adverse-weather/10.1117/12.3062388.short info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023/PID2023-152427OB-I00/ES/SENSOR MULTIMODAL BASADO EN LIDAR PARA SENSADO AMBIENTAL EN MEDIOS SUBMARINOS/ Restricted access - publisher's policy application/pdf International Society for Photo-Optical Instrumentation Engineers (SPIE)