A middleware for developing and deploying scalable remote mining services

Leonid Glimcher, Gagan Agrawal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

In this paper, we consider the problem of developing service-oriented implementations of data-intensive applications that process data on remote servers. While the existing grid and web-service frameworks allow interoperability and flexible resource utilization, achieving efficiency and scalability remains a critical challenge. Similarly, the existing grid and web-service frameworks do not provide transparency in accessing and processing data from grid-based data servers. We present design and evaluation of a system that supports a high-level interface for developing data mining and scientific data processing grid-services and targets data residing on SRB servers. Results of our evaluation using two data mining and one scientific data processing applications show two important observations. First, each of applications we evaluated demonstrated good scalability with respect to dataset size, as well as changing numbers of both data host and compute nodes. Second, there is only a small overhead associated with deploying our middleware-based applications using MPICH-G2 and Globus. This overhead varied between 14% and 22% and is primarily because of a larger memory footprint. Thus, overall, our work shows that it is feasible to develop and deploy scalable and efficient grid-services that process data from remote servers.

Original languageEnglish (US)
Title of host publicationProceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
Pages242-249
Number of pages8
DOIs
StatePublished - 2008
Externally publishedYes
EventCCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid - Lyon, France
Duration: May 19 2008May 22 2008

Publication series

NameProceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid

Conference

ConferenceCCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
CountryFrance
CityLyon
Period5/19/085/22/08

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'A middleware for developing and deploying scalable remote mining services'. Together they form a unique fingerprint.

Cite this