Skip to Main content Skip to Navigation
Theses

Evaluating the impact of streaming systems design on application performance

Abstract : Data Stream Processing (DSP) is an established Big Data paradigm that allows to process and analyze data in real-time. Streaming applications are composed of a series of tasks, replicated and distributed over a cluster, that performs operations on the incoming data, providing continuous results updates. A wide range of works tackled several aspects of DSP, to improve system reliability and performance: task placement, fault tolerance and state management are just some of many examples. In this thesis, we study the limitations of current DSP platforms, focusing on performance from the application point-of-view.In the first part, we analyse message reliability mechanisms in streaming platforms. We uncover the tight interdependency between platform mechanisms and tasks scheduling algorithms. Especially when those mechanisms are implemented as non-functional tasks. Thus, we present two scheduling algorithms to optimize application performance, taking into account the impact of non-functional tasks placement. We show how correctly placing those tasks the performance of the streaming application are improved.In the second part, we present NAMB, an application prototype generator to tackle the shortcomings of current streaming application testing. First, we introduce the fundamentals over which we base NAMB, presenting the high-level description models used to define streaming applications. Then, we illustrate our application prototype generator, detailing the challenges of its implementation. Finally, we perform a wide evaluation, where we illustrate numerous possible use cases for our tool, giving insights on the processing load tuning. We demonstrate NAMB characteristics as a generic and flexible solution.
Complete list of metadata

https://tel.archives-ouvertes.fr/tel-03273377
Contributor : Abes Star :  Contact
Submitted on : Tuesday, June 29, 2021 - 10:58:25 AM
Last modification on : Wednesday, June 30, 2021 - 3:28:12 AM

File

2021COAZ4011.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-03273377, version 1

Collections

Citation

Alessio Pagliari. Evaluating the impact of streaming systems design on application performance. Data Structures and Algorithms [cs.DS]. Université Côte d'Azur, 2021. English. ⟨NNT : 2021COAZ4011⟩. ⟨tel-03273377⟩

Share

Metrics

Record views

60

Files downloads

49