Optimizing Distributed In-memory Storage Systems˸ Fault-tolerance, Performance, Energy Efficiency

Abstract : Emerging technologies such as connected devices and social networking applications are shaping the way we live, work, and interact with each other. These technologies generate increasingly high volumes of data. Dealing with large volumes of data has been an important focus in the last decade, however, today the challenge has shifted from data volume to velocity: How to store, process, and extract value from data generated With the growing capacity of DRAM, service providers largely rely on DRAM-based storage systems to serve their workloads. Because DRAM is volatile, usually, distributed in-memory storage systems rely on expensive durability mechanisms to persist data.This creates trade-offs between performance, durability and efficiency in in-memory storage systems We first study these trade-offs by means of experimental study. We extract the main factors that impact performance and efficiency in in-memory storage systems. Then, we design and implement a new RDMA-based replication mechanism that greatly improves replication efficiency in in-memory storage systems. Finally, we leverage our techniques and apply them to stream storage systems. We design and implement high-performance replication mechanisms for stream storage, while guaranteeing linearizability and durability.
Document type :
Theses
Complete list of metadatas

Cited literature [104 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-01891897
Contributor : Abes Star <>
Submitted on : Monday, October 15, 2018 - 2:22:06 PM
Last modification on : Tuesday, December 3, 2019 - 10:21:04 AM
Long-term archiving on: Wednesday, January 16, 2019 - 2:49:36 PM

File

2018ENSR0015_TALEB_VF.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-01891897, version 2

Citation

Mohammed Yacine Taleb. Optimizing Distributed In-memory Storage Systems˸ Fault-tolerance, Performance, Energy Efficiency. Performance [cs.PF]. École normale supérieure de Rennes, 2018. English. ⟨NNT : 2018ENSR0015⟩. ⟨tel-01891897v2⟩

Share

Metrics

Record views

258

Files downloads

563