Scalable data-management systems for Big Data

Viet-Trung Tran 1
1 KerData - Scalable Storage for Clouds and Beyond
IRISA-D1 - SYSTÈMES LARGE ÉCHELLE, Inria Rennes – Bretagne Atlantique
Abstract : Big Data can be characterized by 3 V’s. •Big Volume refers to the unprecedented growth in the amount of data. •Big Velocity refers to the growth in the speed of moving data in and out management systems. •Big Variety refers to the growth in the number of different data formats. Managing Big Data requires fundamental changes in the architecture of data management systems. Data storage should continue being innovated in order to adapt to the growth of data. They need to be scalable while maintaining high performance regarding data accesses. This thesis focuses on building scalable data management systems for Big Data. Our first and second contributions address the challenge of providing efficient support for Big Volume of data in data-intensive high performance computing (HPC) environments. Particularly, we address the shortcoming of existing approaches to handle atomic, non-contiguous I/O operations in a scalable fashion. We propose and implement a versioning-based mechanism that can be leveraged to offer isolation for non-contiguous I/O without the need to perform expensive synchronizations. In the context of parallel array processing in HPC, we introduce Pyramid, a large-scale, array-oriented storage system. It revisits the physical organization of data in distributed storage systems for scalable performance. Pyramid favors multidimensional-aware data chunking, that closely matches the access patterns generated by applications. Pyramid also favors a distributed metadata management and a versioning concurrency control to eliminate synchronizations in concurrency. Our third contribution addresses Big Volume at the scale of the geographically distributed environments. We consider BlobSeer, a distributed versioning-oriented data management service, and we propose BlobSeer-WAN, an extension of BlobSeer optimized for such geographically distributed environments. BlobSeer-WAN takes into account the latency hierarchy by favoring locally metadata accesses. BlobSeer-WAN features asynchronous metadata replication and a vector-clock implementation for collision resolution. To cope with the Big Velocity characteristic of Big Data, our last contribution feautures DStore, an in-memory document-oriented store that scale vertically by leveraging large memory capability in multicore machines. DStore demonstrates fast and atomic complex transaction processing in data writing, while maintaining high throughput read access. DStore follows a single-threaded execution model to execute update transactions sequentially, while relying on a versioning concurrency control to enable a large number of simultaneous readers.
Document type :
Theses
Liste complète des métadonnées

Cited literature [91 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-00920432
Contributor : Abes Star <>
Submitted on : Wednesday, December 18, 2013 - 2:27:08 PM
Last modification on : Friday, November 16, 2018 - 1:40:31 AM
Document(s) archivé(s) le : Wednesday, March 19, 2014 - 5:50:07 AM

File

Tran2013.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-00920432, version 1

Citation

Viet-Trung Tran. Scalable data-management systems for Big Data. Other [cs.OH]. École normale supérieure de Cachan - ENS Cachan, 2013. English. ⟨NNT : 2013DENS0001⟩. ⟨tel-00920432⟩

Share

Metrics

Record views

1875

Files downloads

3146