Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems

Kishori M. Konwar, N. Prakash, Erez Kantor, Nancy Lynch, Muriel Medard, Alexander Allister Schwarzmann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Erasure codes are increasingly being studied in the context of implementing atomic memory objects in large scale asynchronous distributed storage systems. When compared with the traditional replication based schemes, erasure codes have the potential of significantly lowering storage and communication costs while simultaneously guaranteeing the desired resiliency levels. In this work, we propose the Storage-Optimized Data-Atomic (SODA) algorithm for implementing atomic memory objects in the multi-writer multi-reader setting. SODA uses Maximum Distance Separable (MDS) codes, and is specifically designed to optimize the total storage cost for a given fault-tolerance requirement. For tolerating f server crashes in an n-server system, SODA uses an [n, k] MDS code with k = n - f, and incurs a total storage cost of n/n-f. SODA is designed under the assumption of reliable point-to-point communication channels. The communication cost of a write and a read operation are respectively given by O(f2) and n/n-f(w+1), where w denotes the number of writes that are concurrent with the particular read. In comparison with the recent CASGC algorithm [1], which also uses MDS codes, SODA offers lower storage cost while pays more on the communication cost. We also present a modification of SODA, called SODAerr, to handle the case where some of the servers can return erroneous coded elements during a read operation. Specifically, in order to tolerate f server failures and e error-prone coded elements, the SODAerr algorithm uses an [n, k] MDS code such that k = n - 2e - f. SODAerr also guarantees liveness and atomicity, while maintaining an optimized total storage cost of n/n-f-2e.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages720-729
Number of pages10
ISBN (Electronic)9781509021406
DOIs
StatePublished - Jul 18 2016
Event30th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2016 - Chicago, United States
Duration: May 23 2016May 27 2016

Publication series

NameProceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016

Other

Other30th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2016
CountryUnited States
CityChicago
Period5/23/165/27/16

Fingerprint

Data storage equipment
Servers
Costs
Communication
Fault tolerance
Computer systems

Keywords

  • Atomicity
  • Codes for storage
  • Communication cost
  • Muti-writer multi-reader
  • Storage cost

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Konwar, K. M., Prakash, N., Kantor, E., Lynch, N., Medard, M., & Schwarzmann, A. A. (2016). Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems. In Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016 (pp. 720-729). [7516068] (Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS.2016.55

Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems. / Konwar, Kishori M.; Prakash, N.; Kantor, Erez; Lynch, Nancy; Medard, Muriel; Schwarzmann, Alexander Allister.

Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 720-729 7516068 (Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Konwar, KM, Prakash, N, Kantor, E, Lynch, N, Medard, M & Schwarzmann, AA 2016, Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems. in Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016., 7516068, Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016, Institute of Electrical and Electronics Engineers Inc., pp. 720-729, 30th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2016, Chicago, United States, 5/23/16. https://doi.org/10.1109/IPDPS.2016.55
Konwar KM, Prakash N, Kantor E, Lynch N, Medard M, Schwarzmann AA. Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems. In Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 720-729. 7516068. (Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016). https://doi.org/10.1109/IPDPS.2016.55
Konwar, Kishori M. ; Prakash, N. ; Kantor, Erez ; Lynch, Nancy ; Medard, Muriel ; Schwarzmann, Alexander Allister. / Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems. Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 720-729 (Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016).
@inproceedings{6ba09397aefd40dcaadb3f79de0786e6,
title = "Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems",
abstract = "Erasure codes are increasingly being studied in the context of implementing atomic memory objects in large scale asynchronous distributed storage systems. When compared with the traditional replication based schemes, erasure codes have the potential of significantly lowering storage and communication costs while simultaneously guaranteeing the desired resiliency levels. In this work, we propose the Storage-Optimized Data-Atomic (SODA) algorithm for implementing atomic memory objects in the multi-writer multi-reader setting. SODA uses Maximum Distance Separable (MDS) codes, and is specifically designed to optimize the total storage cost for a given fault-tolerance requirement. For tolerating f server crashes in an n-server system, SODA uses an [n, k] MDS code with k = n - f, and incurs a total storage cost of n/n-f. SODA is designed under the assumption of reliable point-to-point communication channels. The communication cost of a write and a read operation are respectively given by O(f2) and n/n-f(w+1), where w denotes the number of writes that are concurrent with the particular read. In comparison with the recent CASGC algorithm [1], which also uses MDS codes, SODA offers lower storage cost while pays more on the communication cost. We also present a modification of SODA, called SODAerr, to handle the case where some of the servers can return erroneous coded elements during a read operation. Specifically, in order to tolerate f server failures and e error-prone coded elements, the SODAerr algorithm uses an [n, k] MDS code such that k = n - 2e - f. SODAerr also guarantees liveness and atomicity, while maintaining an optimized total storage cost of n/n-f-2e.",
keywords = "Atomicity, Codes for storage, Communication cost, Muti-writer multi-reader, Storage cost",
author = "Konwar, {Kishori M.} and N. Prakash and Erez Kantor and Nancy Lynch and Muriel Medard and Schwarzmann, {Alexander Allister}",
year = "2016",
month = "7",
day = "18",
doi = "10.1109/IPDPS.2016.55",
language = "English (US)",
series = "Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "720--729",
booktitle = "Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016",

}

TY - GEN

T1 - Storage-Optimized Data-Atomic Algorithms for Handling Erasures and Errors in Distributed Storage Systems

AU - Konwar, Kishori M.

AU - Prakash, N.

AU - Kantor, Erez

AU - Lynch, Nancy

AU - Medard, Muriel

AU - Schwarzmann, Alexander Allister

PY - 2016/7/18

Y1 - 2016/7/18

N2 - Erasure codes are increasingly being studied in the context of implementing atomic memory objects in large scale asynchronous distributed storage systems. When compared with the traditional replication based schemes, erasure codes have the potential of significantly lowering storage and communication costs while simultaneously guaranteeing the desired resiliency levels. In this work, we propose the Storage-Optimized Data-Atomic (SODA) algorithm for implementing atomic memory objects in the multi-writer multi-reader setting. SODA uses Maximum Distance Separable (MDS) codes, and is specifically designed to optimize the total storage cost for a given fault-tolerance requirement. For tolerating f server crashes in an n-server system, SODA uses an [n, k] MDS code with k = n - f, and incurs a total storage cost of n/n-f. SODA is designed under the assumption of reliable point-to-point communication channels. The communication cost of a write and a read operation are respectively given by O(f2) and n/n-f(w+1), where w denotes the number of writes that are concurrent with the particular read. In comparison with the recent CASGC algorithm [1], which also uses MDS codes, SODA offers lower storage cost while pays more on the communication cost. We also present a modification of SODA, called SODAerr, to handle the case where some of the servers can return erroneous coded elements during a read operation. Specifically, in order to tolerate f server failures and e error-prone coded elements, the SODAerr algorithm uses an [n, k] MDS code such that k = n - 2e - f. SODAerr also guarantees liveness and atomicity, while maintaining an optimized total storage cost of n/n-f-2e.

AB - Erasure codes are increasingly being studied in the context of implementing atomic memory objects in large scale asynchronous distributed storage systems. When compared with the traditional replication based schemes, erasure codes have the potential of significantly lowering storage and communication costs while simultaneously guaranteeing the desired resiliency levels. In this work, we propose the Storage-Optimized Data-Atomic (SODA) algorithm for implementing atomic memory objects in the multi-writer multi-reader setting. SODA uses Maximum Distance Separable (MDS) codes, and is specifically designed to optimize the total storage cost for a given fault-tolerance requirement. For tolerating f server crashes in an n-server system, SODA uses an [n, k] MDS code with k = n - f, and incurs a total storage cost of n/n-f. SODA is designed under the assumption of reliable point-to-point communication channels. The communication cost of a write and a read operation are respectively given by O(f2) and n/n-f(w+1), where w denotes the number of writes that are concurrent with the particular read. In comparison with the recent CASGC algorithm [1], which also uses MDS codes, SODA offers lower storage cost while pays more on the communication cost. We also present a modification of SODA, called SODAerr, to handle the case where some of the servers can return erroneous coded elements during a read operation. Specifically, in order to tolerate f server failures and e error-prone coded elements, the SODAerr algorithm uses an [n, k] MDS code such that k = n - 2e - f. SODAerr also guarantees liveness and atomicity, while maintaining an optimized total storage cost of n/n-f-2e.

KW - Atomicity

KW - Codes for storage

KW - Communication cost

KW - Muti-writer multi-reader

KW - Storage cost

UR - http://www.scopus.com/inward/record.url?scp=84983301357&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84983301357&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2016.55

DO - 10.1109/IPDPS.2016.55

M3 - Conference contribution

T3 - Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016

SP - 720

EP - 729

BT - Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -