Please use this identifier to cite or link to this item:
Title: Data insertion and archiving in erasure-coding based large-scale storage systems
Authors: Pamies-Juarez, Lluis
Oggier, Frederique
Datta, Anwitaman
Keywords: DRNTU::Engineering::Computer science and engineering::Data::Data storage representations
Issue Date: 2013
Source: Pamies-Juarez, L., Oggier, F., & Datta, A. (2013). Data Insertion and Archiving in Erasure-Coding Based Large-Scale Storage Systems. Distributed Computing and Internet Technology, 7753, pp.47-68.
Abstract: Given the vast volume of data that needs to be stored reliably, many data-centers and large-scale file systems have started using erasure codes to achieve reliable storage while keeping the storage overhead low. This has invigorated the research on erasure codes tailor made to achieve different desirable storage system properties such as efficient redundancy replenishment mechanisms, resilience against data corruption, degraded reads, to name a few prominent ones. A problem that has mainly been overlooked until recently is that of how the storage system can be efficiently populated with erasure coded data to start with. In this paper, we will look at two distinct but related scenarios: (i) migration to archival -leveraging on existing replicated data to create an erasure encoded archive, and (ii) data insertion - new data being inserted in the system directly in erasure coded format. We will elaborate on coding techniques to achieve better throughput for data insertion and migration, and in doing so, explore the connection of these techniques with recently proposed locally repairable codes such as self-repairing codes.
DOI: 10.1007/978-3-642-36071-8_4
Rights: © 2013 Springer. This is the author created version of a work that has been peer reviewed and accepted for publication by Distributed Computing and Internet Technology, Springer. It incorporates referee’s comments but changes resulting from the publishing process, such as copyediting, structural formatting, may not be reflected in this document. The published version is available at: [].
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Conference Papers
SPMS Conference Papers

Files in This Item:
File Description SizeFormat 
icdcitv2.pdf328.82 kBAdobe PDFThumbnail

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.