Para acceder a los documentos con el texto completo, por favor, siga el siguiente enlace: http://hdl.handle.net/2117/104402

The state of SQL-on-Hadoop in the cloud
Poggi, Nicolas; Berral García, Josep Lluís; Fenech, Thomas; Carrera Pérez, David; Blakeley, Jose; Minhas, Umar F.; Vujic, Nikola
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors; Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
Managed Hadoop in the cloud, especially SQL-on-Hadoop, has been gaining attention recently. On Platform-as-a-Service (PaaS), analytical services like Hive and Spark come preconfigured for general-purpose and ready to use. Thus, giving companies a quick entry and on-demand deployment of ready SQL-like solutions for their big data needs. This study evaluates cloud services from an end-user perspective, comparing providers including: Microsoft Azure, Amazon Web Services, Google Cloud, and Rackspace. The study focuses on performance, readiness, scalability, and cost-effectiveness of the different solutions at entry/test level clusters sizes. Results are based on over 15,000 Hive queries derived from the industry standard TPC-H benchmark. The study is framed within the ALOJA research project, which features an open source benchmarking and analysis platform that has been recently extended to support SQL-on-Hadoop engines. The ALOJA Project aims to lower the total cost of ownership (TCO) of big data deployments and study their performance characteristics for optimization. The study benchmarks cloud providers across a diverse range instance types, and uses input data scales from 1GB to 1TB, in order to survey the popular entry-level PaaS SQL-on-Hadoop solutions, thereby establishing a common results-base upon which subsequent research can be carried out by the project. Initial results already show the main performance trends to both hardware and software configuration, pricing, similarities and architectural differences of the evaluated PaaS solutions. Whereas some providers focus on decoupling storage and computing resources while offering network-based elastic storage, others choose to keep the local processing model from Hadoop for high performance, but reducing flexibility. Results also show the importance of application-level tuning and how keeping up-to-date hardware and software stacks can influence performance even more than replicating the on-premises model in the cloud.
This work is partially supported by the Microsoft Azure for Research program, the European Research Council (ERC) under the EUs Horizon 2020 programme (GA 639595), the Spanish Ministry of Education (TIN2015-65316-P), and the Generalitat de Catalunya (2014-SGR-1051).
Peer Reviewed
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
Big data
Cloud computing
Managed Haddop
SQL-on-Hadoop
Platform-as-a-Service (PaaS)
ALOJA
Macrodades
Computació en núvol
info:eu-repo/semantics/submittedVersion
info:eu-repo/semantics/conferenceObject
Institute of Electrical and Electronics Engineers (IEEE)
         

Mostrar el registro completo del ítem

Documentos relacionados

Otros documentos del mismo autor/a

Poggi Mastrokalo, Nicolas; Carrera Pérez, David; Call, Aaron; Mendoza, Sergio; Becerra Fontal, Yolanda; Torres Viñals, Jordi; Ayguadé Parra, Eduard; Gagliardi, Fabrizio; Labarta Mancho, Jesús José; Reinauer, Rob; Vujic, Nikola; Green, Daron; Blakeley, Jose
Berral García, Josep Lluís; Poggi, Nicolas; Carrera Pérez, David; Call, Aaaron; Reinauer, Rob; Green, Daron
Poggi, Nicolas; Berral García, Josep Lluís; Carrera Pérez, David
Velasco Esteban, Luis Domingo; Asensio Garcia, Adrian; Castro Casales, Alberto; Berral García, Josep Lluís; Carrera Pérez, David; López, Víctor; Fernández Palacios, Juan Pedro
Berral García, Josep Lluís; Poggi Mastrokalo, Nicolas; Carrera Pérez, David; Call, Aaron; Reinauer, Rob; Green, Daron