Resource allocation for distributed streaming applications

Zhu Qian, Gagan Agrawal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

We consider resource allocation for distributed streaming applications running In a grid environment, where continuously streaming data needs to be aggregated and processed to produce output streams. Because such an application comprises a pipeline of processing stages, both communication and computational requirements need to be taken Into account while performing resource allocation. In this paper, we give a rigorous formulation of this resource allocation problem, based on the DAG representation of the application as well as the environment. We have shown how we can use the notion of subgraph Isomorphism and developed an effective resource allocation algorithm. The main observations from the experiments we conducted to evaluate our algorithms were as follows: the overhead caused by our algorithm Is comparable to an existing algorithm, Streamline, which Is based on heuristics. At the same time, the application performance was Improved by 30% on average. When compared to the allocation performed by the Optimal algorithm, which enumerates all mappings, the application performance with our algorithm was within 4%. At the same time, unlike the Optimal algorithm, our algorithm scaled well to large graphs.

Original languageEnglish (US)
Title of host publicationProceedings - 37th International Conference on Parallel Processing, ICPP 2008
Pages414-421
Number of pages8
DOIs
StatePublished - 2008
Externally publishedYes
Event37th International Conference on Parallel Processing, ICPP 2008 - Portland, OR, United States
Duration: Sep 9 2008Sep 12 2008

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918

Conference

Conference37th International Conference on Parallel Processing, ICPP 2008
CountryUnited States
CityPortland, OR
Period9/9/089/12/08

ASJC Scopus subject areas

  • Software
  • Mathematics(all)
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Resource allocation for distributed streaming applications'. Together they form a unique fingerprint.

Cite this