Para acceder a los documentos con el texto completo, por favor, siga el siguiente enlace: http://hdl.handle.net/2117/28187

Adaptive MapReduce scheduling in shared environments
Polo, Jordà; Becerra Fontal, Yolanda; Carrera Pérez, David; Torres Viñals, Jordi; Ayguadé Parra, Eduard; Steinder, Malgorzata
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors; Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
In this paper we present a MapReduce task scheduler for shared environments in which MapReduce is executed along with other resource-consuming workloads, such as transactional applications. All workloads may potentially share the same data store, some of them consuming data for analytics purposes while others acting as data generators. This kind of scenario is becoming increasingly important in data centers where improved resource utilization can be achieved through workload consolidation, and is specially challenging due to the interaction between workloads of different nature that compete for limited resources. The proposed scheduler aims to improve resource utilization across machines while observing completion time goals. Unlike other MapReduce schedulers, our approach also takes into account the resource demands for non-MapReduce workloads, and assumes that the amount of resources made available to the MapReduce applications is variable over time. As shown in our experiments, our proposal improves the management of MapReduce jobs in the presence of variable resource availability, increasing the accuracy of the estimations made by the scheduler, thus improving completion time goals without an impact on the fairness of the scheduler.
This work is partially supported by the Ministry of Science and Technology of Spain and the European Union’s FEDER funds (TIN2012-34557), by the Generalitat de Catalunya (2009-SGR-980) by the BSC-CNS Severo Ochoa program (SEV-2011-00067) and by the by the European Commission's IST activity of the 7th Framework Program under contract number 317862 (COMPOSE)
Peer Reviewed
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
Data processing service centers
Parallel programming (Computer science)
Adaptive
Analytics
Availability
Distributed
MapReduce
Scheduling
Shared environments
Transactional
Centres informàtics
Programació en paral·lel (Informàtica)
info:eu-repo/semantics/publishedVersion
info:eu-repo/semantics/conferenceObject
Institute of Electrical and Electronics Engineers (IEEE)
         

Mostrar el registro completo del ítem

Documentos relacionados

Otros documentos del mismo autor/a

Polo, Jordà; Becerra Fontal, Yolanda; Carrera Pérez, David; Torres Viñals, Jordi; Ayguadé Parra, Eduard; Spreitzer, Mike; Steinder, Malgorzata
Carrera Pérez, David; Steinder, Malgorzata; Whalley, Ian; Torres Viñals, Jordi; Ayguadé Parra, Eduard
Carrera Pérez, David; Steinder, Malgorzata; Whalley, Ian; Torres Viñals, Jordi; Ayguadé Parra, Eduard
Carrera Pérez, David; Steinder, Malgorzata; Whalley, Ian; Torres Viñals, Jordi; Ayguadé Parra, Eduard
Carrera Pérez, David; Steinder, Malgorzata; Whalley, Ian; Torres Viñals, Jordi; Ayguadé Parra, Eduard