Task decomposition for adaptive data staging in workflows for distributed environments

Onyeka Ezenwoye, Balaji Viswanathan, S. Masoud Sadjadi, Liana Fong, Gargi Dasgupta, Selim Kalayci

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Scientific workflows are often composed by scientists that are not particularly familiar with performance and fault-tolerance issues of the underlying layer. The inherent nature of the infrastructure and environment for scientific workflow applications means that the movement of data comes with reliability challenges. Improving the reliablility scientific workflows in distributed environments, calls for the decoupling of data staging and computation activities, and each aspect needs to be addressed separately In this paper, we present an approach to managing scientific workflows that specifically provides constructs for reliable data staging. In our framework, data staging tasks are automatically separated from computation tasks in the definition of the workflow. High-level policies can be provided that allow for dynamic adaptation of the workflow to occur. Our approach permits the separate specification of the functional and non-functional requirements of the application and is dynamic enough to allow for the alteration of the workflow at runtime for optimization.

Original languageEnglish (US)
Title of host publicationProceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009
Pages16-19
Number of pages4
StatePublished - Dec 1 2009
Event21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009 - Boston, MA, United States
Duration: Jul 1 2009Jul 3 2009

Publication series

NameProceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009

Other

Other21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009
CountryUnited States
CityBoston, MA
Period7/1/097/3/09

Fingerprint

Decomposition
Fault tolerance
Specifications

Keywords

  • Data staging
  • Distributed systems
  • Scientific workflow

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications

Cite this

Ezenwoye, O., Viswanathan, B., Sadjadi, S. M., Fong, L., Dasgupta, G., & Kalayci, S. (2009). Task decomposition for adaptive data staging in workflows for distributed environments. In Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009 (pp. 16-19). (Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009).

Task decomposition for adaptive data staging in workflows for distributed environments. / Ezenwoye, Onyeka; Viswanathan, Balaji; Sadjadi, S. Masoud; Fong, Liana; Dasgupta, Gargi; Kalayci, Selim.

Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009. 2009. p. 16-19 (Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ezenwoye, O, Viswanathan, B, Sadjadi, SM, Fong, L, Dasgupta, G & Kalayci, S 2009, Task decomposition for adaptive data staging in workflows for distributed environments. in Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009. Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009, pp. 16-19, 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009, Boston, MA, United States, 7/1/09.
Ezenwoye O, Viswanathan B, Sadjadi SM, Fong L, Dasgupta G, Kalayci S. Task decomposition for adaptive data staging in workflows for distributed environments. In Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009. 2009. p. 16-19. (Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009).
Ezenwoye, Onyeka ; Viswanathan, Balaji ; Sadjadi, S. Masoud ; Fong, Liana ; Dasgupta, Gargi ; Kalayci, Selim. / Task decomposition for adaptive data staging in workflows for distributed environments. Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009. 2009. pp. 16-19 (Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009).
@inproceedings{836c20da2cae47ca9bb2f082fdd790e2,
title = "Task decomposition for adaptive data staging in workflows for distributed environments",
abstract = "Scientific workflows are often composed by scientists that are not particularly familiar with performance and fault-tolerance issues of the underlying layer. The inherent nature of the infrastructure and environment for scientific workflow applications means that the movement of data comes with reliability challenges. Improving the reliablility scientific workflows in distributed environments, calls for the decoupling of data staging and computation activities, and each aspect needs to be addressed separately In this paper, we present an approach to managing scientific workflows that specifically provides constructs for reliable data staging. In our framework, data staging tasks are automatically separated from computation tasks in the definition of the workflow. High-level policies can be provided that allow for dynamic adaptation of the workflow to occur. Our approach permits the separate specification of the functional and non-functional requirements of the application and is dynamic enough to allow for the alteration of the workflow at runtime for optimization.",
keywords = "Data staging, Distributed systems, Scientific workflow",
author = "Onyeka Ezenwoye and Balaji Viswanathan and Sadjadi, {S. Masoud} and Liana Fong and Gargi Dasgupta and Selim Kalayci",
year = "2009",
month = "12",
day = "1",
language = "English (US)",
isbn = "1891706241",
series = "Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009",
pages = "16--19",
booktitle = "Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009",

}

TY - GEN

T1 - Task decomposition for adaptive data staging in workflows for distributed environments

AU - Ezenwoye, Onyeka

AU - Viswanathan, Balaji

AU - Sadjadi, S. Masoud

AU - Fong, Liana

AU - Dasgupta, Gargi

AU - Kalayci, Selim

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Scientific workflows are often composed by scientists that are not particularly familiar with performance and fault-tolerance issues of the underlying layer. The inherent nature of the infrastructure and environment for scientific workflow applications means that the movement of data comes with reliability challenges. Improving the reliablility scientific workflows in distributed environments, calls for the decoupling of data staging and computation activities, and each aspect needs to be addressed separately In this paper, we present an approach to managing scientific workflows that specifically provides constructs for reliable data staging. In our framework, data staging tasks are automatically separated from computation tasks in the definition of the workflow. High-level policies can be provided that allow for dynamic adaptation of the workflow to occur. Our approach permits the separate specification of the functional and non-functional requirements of the application and is dynamic enough to allow for the alteration of the workflow at runtime for optimization.

AB - Scientific workflows are often composed by scientists that are not particularly familiar with performance and fault-tolerance issues of the underlying layer. The inherent nature of the infrastructure and environment for scientific workflow applications means that the movement of data comes with reliability challenges. Improving the reliablility scientific workflows in distributed environments, calls for the decoupling of data staging and computation activities, and each aspect needs to be addressed separately In this paper, we present an approach to managing scientific workflows that specifically provides constructs for reliable data staging. In our framework, data staging tasks are automatically separated from computation tasks in the definition of the workflow. High-level policies can be provided that allow for dynamic adaptation of the workflow to occur. Our approach permits the separate specification of the functional and non-functional requirements of the application and is dynamic enough to allow for the alteration of the workflow at runtime for optimization.

KW - Data staging

KW - Distributed systems

KW - Scientific workflow

UR - http://www.scopus.com/inward/record.url?scp=77956299010&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956299010&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:77956299010

SN - 1891706241

SN - 9781891706240

T3 - Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009

SP - 16

EP - 19

BT - Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering, SEKE 2009

ER -