Indexing and parallel query processing support for visualizing climate datasets

Yu Su, Gagan Agrawal, Jonathan Woodring

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

With increasing emphasis on analysis of large-scale scientific data, and with growing dataset sizes, a number of new challenges are arising. Particularly, novel data management solutions are needed, which can work together with the existing tools. This paper examines indexing support for supporting high-level queries (primarily those for sub setting) on array-based scientific datasets. This work is motivated by the limitations arising in visualizing climate datasets (stored in Net CDF), using tools like Para View. We have developed a new indexing strategy, which can help support a variety of sub setting queries over these datasets, including those requiring sub setting over dimensions/coordinates and those involving variable values. Our approach is based on bitmaps, but involves use of two-level indices and careful partitioning, based on query profiles. We also show how our indexing support can be used for sub setting operations executed in parallel. We compare our solutions against a number of other solutions, and demonstrate that our method is more effective.

Original languageEnglish (US)
Title of host publicationProceedings - 41st International Conference on Parallel Processing, ICPP 2012
Pages249-258
Number of pages10
DOIs
StatePublished - 2012
Externally publishedYes
Event41st International Conference on Parallel Processing, ICPP 2012 - Pittsburgh, PA, United States
Duration: Sep 10 2012Sep 13 2012

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918

Conference

Conference41st International Conference on Parallel Processing, ICPP 2012
CountryUnited States
CityPittsburgh, PA
Period9/10/129/13/12

ASJC Scopus subject areas

  • Software
  • Mathematics(all)
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Indexing and parallel query processing support for visualizing climate datasets'. Together they form a unique fingerprint.

Cite this