Exécution interactive pour expériences computationnelles à grande échelle

Jonas Dias 1, 2, 3
Abstract : To tackle the exploratory nature of science and the dynamic process involved in scientific analysis, dynamic workflows have been identified as an open challenge as they are subject to continuous adaptation and improvement. In particular, they require the ability of adapting a scientific workflow, at runtime, based on external events such as human interaction. Supporting dynamic iteration is an important step towards dynamic workflows since user interaction with a workflow is iterative. However, current support for iteration in scientific workflows is static and does not allow for runtime changes in data such as filter criteria or error thresholds. In this thesis, we propose an algebraic approach to support data-centric iteration in dynamic workflows and a dynamic execution model for these operators. We introduce the concept of iteration lineage so that provenance data management is consistent with dynamic changes in the workflow. Lineage also enables scientists to interact with workflow data and configuration at runtime through two steering algorithms implemented in Chiron. We evaluate our approach using real large-scale workflows on a large-scale environment. The results show execution time savings up to 24 days when compared to a traditional non-iterative workflow execution. We also perform complex queries for partial result analysis along the iterations and we assess the max overhead introduced by our iterative model as 3.63% of execution time. The performance of our proposed steering algorithms run in less than 1 millisecond in the worst-case scenario we measured.
Complete list of metadatas

https://tel.archives-ouvertes.fr/tel-00939266
Contributor : Patrick Valduriez <>
Submitted on : Thursday, January 30, 2014 - 2:55:27 PM
Last modification on : Thursday, May 24, 2018 - 3:59:21 PM
Long-term archiving on : Monday, May 5, 2014 - 11:11:43 AM

Identifiers

  • HAL Id : tel-00939266, version 1

Collections

Citation

Jonas Dias. Exécution interactive pour expériences computationnelles à grande échelle. Distributed, Parallel, and Cluster Computing [cs.DC]. Universidade Federal de Rio de Janeiro, 2013. Portuguese. ⟨tel-00939266⟩

Share

Metrics

Record views

851

Files downloads

1123