The evolution of data storage architectures: examining the secure value of the Data Lakehouse
Эволюция архитектур хранения данных: исследование ценности с точки зрения безопасности архитектуры Data Lakehouse
2024-08-15
SCID: 54.1/j2ecnjx3
Discuss with AI
Data Lakehousedata lakedata storage architecturesdata warehousesystematic literature review
Figures from the paper
Abstract (AI)
Abstract The digital shift in society is making continuous growth of data. However, choosing a suitable storage architecture to efficiently store, process, and manage data from numerous sources remains a challenge. Currently, there are three storage architecture generations in practice, and the most recent one is Data Lakehouse. Given its novelty, limited research has been done into the rationale behind its introduction, strengths, and weaknesses. In order to fill this gap, this study aims to investigate the secure value (comparative strengths) of the data lakehouse architecture compared to data warehouse and data lake architectures. After conducting a comprehensive systematic literature review, we propose a data storage evolution model showing the comparative strengths and weaknesses of data warehouse, lake, and lakehouse architectures. With the use of the proposed model and expert interviews, this study demonstrates the secure value of the data lakehouse compared to the preceding architectures. In addition, the study presents a high-level view of the overlapping strengths of data Lakehouse with both data warehouse and data lake. In essence, the artifact produced by this study can be used to explain the rationale behind the evolution of data storage architectures. Further, the proposed model will help the practitioners in studying the trade-off between different architectures to offer recommendations. Finally, authors acknowledge that this study has several limitations, such as the limited sample size for the interviews and the bias due to the use of qualitative research approach. However, all the available measures were taken to minimize the effects of these limitations.
Key Findings
1
Systematic literature review and expert interviews demonstrate the comparative secure value of the data lakehouse over preceding warehouse and lake architectures.
2
The data lakehouse combines overlapping strengths of data warehouses and data lakes, clarifying its rationale as the newest storage architecture generation.
3
The findings are limited by the small interview sample and potential bias inherent in the qualitative research approach.
4
The proposed model helps practitioners evaluate architectural trade-offs and formulate recommendations for selecting among storage architectures.
5
The study develops a data storage evolution model comparing the strengths and weaknesses of data warehouses, data lakes, and data lakehouses.
Research Object
Data storage architectures, specifically data warehouse, data lake, and data lakehouse architectures
Research Subject
The comparative secure value, strengths, weaknesses, and evolutionary relationships among data warehouse, data lake, and data lakehouse architectures
Publication Details
Publication Date
2024-08-15
Journal
Publisher
ISSN
Cited by
31
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest