Task decomposition for adaptive data staging in workflows for distributed environments

Onyeka Ezenwoye, Balaji Viswanathan, S. Masoud Sadjadi, Liana Fong, Gargi Dasgupta, Selim Kalayci

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Scientific workflows are often composed by scientists that are not particularly familiar with performance and fault-tolerance issues of the underlying layer. The inherent nature of the infrastructure and environment for scientific workflow applications means that the movement of data comes with reliability challenges. Improving the reliablility scientific workflows in distributed environments, calls for the decoupling of data staging and computation activities, and each aspect needs to be addressed separately In this paper, we present an approach to managing scientific workflows that specifically provides constructs for reliable data staging. In our framework, data staging tasks are automatically separated from computation tasks in the definition of the workflow. High-level policies can be provided that allow for dynamic adaptation of the workflow to occur. Our approach permits the separate specification of the functional and non-functional requirements of the application and is dynamic enough to allow for the alteration of the workflow at runtime for optimization.

Original languageEnglish (US)
Title of host publicationProceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009
Pages16-19
Number of pages4
Publication statusPublished - Dec 1 2009
Event21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009 - Boston, MA, United States
Duration: Jul 1 2009Jul 3 2009

Publication series

NameProceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009

Other

Other21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009
CountryUnited States
CityBoston, MA
Period7/1/097/3/09

    Fingerprint

Keywords

  • Data staging
  • Distributed systems
  • Scientific workflow

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications

Cite this

Ezenwoye, O., Viswanathan, B., Sadjadi, S. M., Fong, L., Dasgupta, G., & Kalayci, S. (2009). Task decomposition for adaptive data staging in workflows for distributed environments. In Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009 (pp. 16-19). (Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009).