Skip to Main content Skip to Navigation

Improving skewed data dissemination in structured overlays

Maeva Antoine 1
1 SCALE - Safe Composition of Autonomous applications with Large-SCALE Execution environment
Laboratoire I3S - COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
Abstract : Many distributed systems face the problem of load imbalance between machines. With the advent of Big Data, large datasets whose values are often highly skewed are produced by heterogeneous sources to be often processed in real time. Thus, it is necessary to be able to adapt to the variations of size/content/source of the incoming data. In this thesis, we focus on RDF data, a format of the Semantic Web. We propose a novel approach to improve data distribution, based on the use of several order-preserving hash functions. This allows an overloaded peer to independently modify its hash function in order to reduce the interval of values it is responsible for. More generally, to address the load imbalance issue, there exist almost as many load balancing strategies as there are different systems. We show that many load balancing schemes are comprised of the same basic elements, and only the implementation and interconnection of these elements vary. Based on this observation, we describe the concepts behind the building of a common API to implement any load balancing strategy independently from the rest of the code. Implemented on our distributed storage system, the API has a minimal impact on the business code and allows the developer to change only a part of a strategy without modifying the other components. We also show how modifying some parameters can lead to significant improvements in terms of results.
Document type :
Complete list of metadatas

Cited literature [75 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Wednesday, December 16, 2015 - 4:22:47 PM
Last modification on : Tuesday, January 12, 2021 - 8:44:01 AM
Long-term archiving on: : Thursday, March 17, 2016 - 3:11:05 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01245077, version 1



Maeva Antoine. Improving skewed data dissemination in structured overlays. Other [cs.OH]. Université Nice Sophia Antipolis, 2015. English. ⟨NNT : 2015NICE4054⟩. ⟨tel-01245077⟩



Record views


Files downloads