Scalability and Optimisation of GroupBy-Joins in MapReduce Scalability and Optimisation of GroupBy-Joins in MapReduceReportar como inadecuado




Scalability and Optimisation of GroupBy-Joins in MapReduce Scalability and Optimisation of GroupBy-Joins in MapReduce - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.

1 LIFO - Laboratoire d-Informatique Fondamentale d-Orléans 2 Lebanese International University, Beirut, Lebanon

Abstract : For over a decade, MapReduce has become the leading programming model for parallel and massive processing of large volumes of data. This has been driven by the development of many frameworks such as Spark, Pig and Hive, facilitating data analysis on large-scale systems. However, these frameworks still remain vulnerable to communication costs, data skew and tasks imbalance problems. This can have a devastating effect on the performance and on the scalability of these systems, more particularly when treating GroupBy-Join queries of large datasets. In this paper, we present a new GroupBy-Join algorithm allowing to reduce communication costs considerably while avoiding data skew effects. A cost analysis of this algorithm shows that our approach is insensitive to data skew and ensures perfect balancing properties during all stages of GroupBy-Join computation even for highly skewed data. These performances have been confirmed by a series of experimentations.

Keywords : Join and GroupBy-join operations Data skew MapReduce programming model Distributed file systems Hadoop framework Apache Pig Latin





Autor: Mostafa Bamha - Mohamad Al Hajj Hassan -

Fuente: https://hal.archives-ouvertes.fr/



DESCARGAR PDF




Documentos relacionados