Título:
|
Spark deployment and performance evaluation on the MareNostrum supercomputer
|
Autor/a:
|
Tous Liesa, Rubén; Gounaris, Anastasios; Tripiana, Carlos; Torres Viñals, Jordi; Girona Turell, Sergi; Ayguadé Parra, Eduard; Labarta Mancho, Jesús José; Becerra Fontal, Yolanda; Carrera Pérez, David; Valero Cortés, Mateo
|
Otros autores:
|
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors; Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions |
Abstract:
|
In this paper we present a framework to enable data-intensive Spark workloads on MareNostrum, a petascale supercomputer designed mainly for compute-intensive applications. As far as we know, this is the first attempt to investigate optimized deployment configurations of Spark on a petascale HPC setup. We detail the design of the framework and present some benchmark data to provide insights into the scalability of the system. We examine the impact of different configurations including parallelism, storage and networking alternatives, and we discuss several aspects in executing Big Data workloads on a computing system that is based on the compute-centric paradigm. Further, we derive conclusions aiming to pave the way towards systematic and optimized methodologies for fine-tuning data-intensive application on large clusters emphasizing on parallelism configurations. |
Abstract:
|
Peer Reviewed |
Materia(s):
|
-Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors -Big data -Parallel processing (Electronic computers) -Sparks -Benchmark testing -Supercomputers -Scalability -Heart beat -Macrodades -Processament en paral·lel (Ordinadors) |
Derechos:
|
|
Tipo de documento:
|
Artículo - Versión presentada Objeto de conferencia |
Editor:
|
Institute of Electrical and Electronics Engineers (IEEE)
|
Compartir:
|
|