2026-04-17T02:37:58Zhttps://recercat.cat/oai/request
oai:recercat.cat:2072/4731452024-11-04T07:10:55Zcom_2072_98col_2072_378192
RECERCAT author Acevedo Giménez, César Esteban author Hernández Budé, Porfidio author Espinosa, Antonio author Méndez Muñoz, Víctor 2024-11-04T07:10:55Z 2024-11-04T07:10:55Z 2017 http://hdl.handle.net/2072/473145 A representative set of workflows found in bioinformatics pipelines must deal with large data sets. Most scientific workflows are defined as Direct Acyclic Graphs (DAGs). Despite DAGs are useful to understand dependence relationships, they do not provide any information about input, output and temporal data files. This information about the location of files of data intensive applications helps to avoid performance issues. This paper presents a multiworkflow store-aware scheduler in a cluster environment called Critical Path File Location (CPFL) policy where the access time to disk is more relevant than network, as an extension of the classical list scheduling policies. Our purpose is to find the best location of data files in a hierarchical storage system. The resulting algorithm is tested in an HPC cluster and in a simulated cluster scenario with bioinformatics synthetic workflows, and largely used benchmarks like Montage and Epigenomics. The resulting simulator is tuned and validated with the first test results from the real infrastructure. The evaluation of our proposal shows promising results up to 70% on benchmarks in real HPC clusters using 128 cores and up to 69% of makespan improvement on simulated 512 cores clusters with a deviation between 0.9% and 3% regarding the real HPC cluster. open access Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, i la comunicació pública de l'obra, sempre que no sigui amb finalitats comercials, i sempre que es reconegui l'autoria de l'obra original. No es permet la creació d'obres derivades. https://creativecommons.org/licenses/by-nc-nd/4.0/ Multiworkflows Cluster Scheduler Simulation Critical path Data processing A Critical Path File Location (CPFL) algorithm for data-aware multiworkflow scheduling on HPC clusters Article