Design and implementation of a fault tolerant job flow manager using job flow patterns and recovery policies

Selim Kalayci, Onyeka Ezenwoye, Balaji Viswanathan, Gargi Dasgupta, S. Masoud Sadjadi, Liana Fong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

Currently, many grid applications are developed as job flows that are composed of multiple jobs. The execution of job flows requires the support of a job flow manager and a job scheduler. Due to the long running nature of job flows, the support for fault tolerance and recovery policies is especially important. This support is inherently complicated due to the sequencing and dependency of jobs within a flow, and the required coordination between workflow engines and job schedulers. In this paper, we describe the design and implementation of a job flow manager that supports fault tolerance. First, we identify and label job flow patterns within a job flow during deployment time. Next, at runtime, we introduce a proxy that intercepts and resolves faults using job flow patterns and their corresponding fault-recovery policies. Our design has the advantages of separation of the job flow and fault handling logic, requiring no manipulation at the modeling time, and providing flexibility with respect to fault resolution at runtime. We validate our design with a prototypical implementation based on the ActiveBPEL workflow engine and GridWay Meta-scheduler, and Montage application as the case study.

Original languageEnglish (US)
Title of host publicationService-Oriented Computing - ICSOC 2008 - 6th International Conference, Proceedings
Pages54-69
Number of pages16
DOIs
StatePublished - Dec 31 2008
Externally publishedYes
Event6th International Conference on Service-Oriented Computing, ICSOC 2008 - Sydney, Australia
Duration: Dec 1 2008Dec 5 2008

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5364 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other6th International Conference on Service-Oriented Computing, ICSOC 2008
CountryAustralia
CitySydney
Period12/1/0812/5/08

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Design and implementation of a fault tolerant job flow manager using job flow patterns and recovery policies'. Together they form a unique fingerprint.

Cite this