The Data Lakehouse: Data Warehousing and More
Озеро-склад данных: хранилище данных и не только
2023-10-12
SCID: 54.1/ymt5rnrp
Discuss with AI
RDBMS-OLAPcloud data lakesdata lakehousedata warehousingvendor lock-in
Figures from the paper
Abstract (AI)
Relational Database Management Systems designed for Online Analytical Processing (RDBMS-OLAP) have been foundational to democratizing data and enabling analytical use cases such as business intelligence and reporting for many years. However, RDBMS-OLAP systems present some well-known challenges. They are primarily optimized only for relational workloads, lead to proliferation of data copies which can become unmanageable, and since the data is stored in proprietary formats, it can lead to vendor lock-in, restricting access to engines, tools, and capabilities beyond what the vendor offers. As the demand for data-driven decision making surges, the need for a more robust data architecture to address these challenges becomes ever more critical. Cloud data lakes have addressed some of the shortcomings of RDBMS-OLAP systems, but they present their own set of challenges. More recently, organizations have often followed a two-tier architectural approach to take advantage of both these platforms, leveraging both cloud data lakes and RDBMS-OLAP systems. However, this approach brings additional challenges, complexities, and overhead. This paper discusses how a data lakehouse, a new architectural approach, achieves the same benefits of an RDBMS-OLAP and cloud data lake combined, while also providing additional advantages. We take today's data warehousing and break it down into implementation independent components, capabilities, and practices. We then take these aspects and show how a lakehouse architecture satisfies them. Then, we go a step further and discuss what additional capabilities and benefits a lakehouse architecture provides over an RDBMS-OLAP.
Key Findings
1
Beyond matching conventional data-warehouse and data-lake capabilities, lakehouses provide additional capabilities and benefits compared with RDBMS-OLAP systems.
2
Cloud data lakes address some RDBMS-OLAP shortcomings but introduce distinct challenges, prompting organizations to adopt complex two-tier architectures combining both platforms.
3
RDBMS-OLAP systems face limitations including relational-workload specialization, proliferation of data copies, proprietary formats, and vendor lock-in.
4
The data lakehouse is presented as an architectural approach intended to unify the benefits of cloud data lakes and RDBMS-OLAP systems while reducing duplication, complexity, and operational overhead.
5
The paper decomposes data warehousing into implementation-independent components, capabilities, and practices, then demonstrates how lakehouse architecture satisfies them.
Research Object
data lakehouse architecture
Research Subject
the capabilities, benefits, and architectural properties of a data lakehouse compared with RDBMS-OLAP systems and cloud data lakes
Publication Details
Publication Date
2023-10-12
Journal
Publisher
ISSN
Cited by
38
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest