dc.contributor
Universitat Politècnica de Catalunya. Departament d'Enginyeria Minera, Industrial i TIC
dc.contributor
Barcelona Supercomputing Center
dc.contributor
Universitat Politècnica de Catalunya. CROMAI - Computing Resources Orchestration and Management for AI
dc.contributor.author
Benavides Arévalo, Andrés
dc.contributor.author
Coll Tejeda, Daniel
dc.contributor.author
Call Barreiro, Erin
dc.contributor.author
Garcia López, Pedro
dc.contributor.author
Nou Castell, Ramon
dc.identifier
Benavides, A. [et al.]. Enhancing HPC with serverless computing: Lithops on MareNostrum5. A: IEEE International Conference on Network Protocols. «2024 IEEE 32nd International Conference on Network Protocols (ICNP 2024): Charleroi, Belgium, October 28-31, 2024». Institute of Electrical and Electronics Engineers (IEEE), 2024. ISBN 979-8-3503-5171-2. DOI 10.1109/ICNP61940.2024.10858564 .
dc.identifier
979-8-3503-5171-2
dc.identifier
https://hdl.handle.net/2117/446227
dc.identifier
10.1109/ICNP61940.2024.10858564
dc.description.abstract
Serverless computing offers a novel alternative for developing and deploying applications. By abstracting backend architecture from the user, developers are encouraged to write code without worrying about server management, scaling, or maintenance. The Function-as-a-Service (FaaS) model optimizes this by allowing developers to deploy discrete functions that scale automatically in response to demand without deep concerns about the scalability of the execution infrastructure or platform. Lithops, a multi-cloud serverless computing framework, follows this trend and enables developers to execute Python code across thousands of cloud cores without modifying local scripts. Despite its potential to deploy parallel jobs, Lithops has been designed for running big data jobs over cloud environments, and its applicability to High-Performance Computing (HPC) systems has been unexplored. This paper introduces a novel architecture enabling Lithops deployment in HPC systems like the MareNostrum 5 supercomputer. By leveraging the immense computational power of the HPC-MN5 supercomputer and the FaaS model of Lithops, the architecture aims to offer high performance and scalability while simplifying application coding and deployment. Our evaluations display Lithops' benchmarks over the MareNostrum 5 HPC scale with the number of nodes, outperforming other commercial cloud platforms in terms of Floating Point Operations Per Second (FLOPS) and read-write bandwidth, and avoiding CPU wastage.
dc.description.abstract
This work has been partially financed by the European Commission (EU-HORIZON NEARDATA GA 101092644), Ministry of Science and Innovation under grant MCIN AEI/10.13039/ 501100011033/FEDER and PID GA PID2019- 107255GB-C21, and the Generalitat de Catalunya (AGAUR) under grant agreements 2021-SGR-00478 and 2021-SGR01626. The authors thankfully acknowledge RES resources provided by Barcelona Supercomputing Center in Data-Centric Computing Group to BCV-2024-2-0004.
dc.description.abstract
Peer Reviewed
dc.description.abstract
Postprint (published version)
dc.format
application/pdf
dc.publisher
Institute of Electrical and Electronics Engineers (IEEE)
dc.relation
https://ieeexplore.ieee.org/document/10858564/
dc.relation
info:eu-repo/grantAgreement/EC/HE/101092644/EU/Extreme Near-Data Processing Platform/NEARDATA
dc.relation
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2019-107255GB-C21/ES/BSC - COMPUTACION DE ALTAS PRESTACIONES VIII/
dc.rights
Restricted access - publisher's policy
dc.subject
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject
Computational modeling
dc.subject
Serverless computing
dc.subject
Computer architecture
dc.subject
Benchmark testing
dc.subject
Market research
dc.subject
Supercomputers
dc.subject
Function-as-a-Service
dc.subject
High performance computing
dc.title
Enhancing HPC with serverless computing: Lithops on MareNostrum5
dc.type
Conference report
