The Data Lakehouse: Data Warehousing and More

Озеро-склад данных: хранилище данных и не только
Dipankar Mazumdar, Hughes, Jason, JB Onofre
2023-10-12

RDBMS-OLAPcloud data lakesdata lakehousedata warehousingvendor lock-in
Relational Database Management Systems designed for Online Analytical Processing (RDBMS-OLAP) have been foundational to democratizing data and enabling analytical use cases such as business intelligence and reporting for many years. However, RDBMS-OLAP systems present some well-known challenges. They are primarily optimized only for relational workloads, lead to proliferation of data copies which can become unmanageable, and since the data is stored in proprietary formats, it can lead to vendor lock-in, restricting access to engines, tools, and capabilities beyond what the vendor offers. As the demand for data-driven decision making surges, the need for a more robust data architecture to address these challenges becomes ever more critical. Cloud data lakes have addressed some of the shortcomings of RDBMS-OLAP systems, but they present their own set of challenges. More recently, organizations have often followed a two-tier architectural approach to take advantage of both these platforms, leveraging both cloud data lakes and RDBMS-OLAP systems. However, this approach brings additional challenges, complexities, and overhead. This paper discusses how a data lakehouse, a new architectural approach, achieves the same benefits of an RDBMS-OLAP and cloud data lake combined, while also providing additional advantages. We take today's data warehousing and break it down into implementation independent components, capabilities, and practices. We then take these aspects and show how a lakehouse architecture satisfies them. Then, we go a step further and discuss what additional capabilities and benefits a lakehouse architecture provides over an RDBMS-OLAP.
1
Beyond matching conventional data-warehouse and data-lake capabilities, lakehouses provide additional capabilities and benefits compared with RDBMS-OLAP systems.
2
Cloud data lakes address some RDBMS-OLAP shortcomings but introduce distinct challenges, prompting organizations to adopt complex two-tier architectures combining both platforms.
3
RDBMS-OLAP systems face limitations including relational-workload specialization, proliferation of data copies, proprietary formats, and vendor lock-in.
4
The data lakehouse is presented as an architectural approach intended to unify the benefits of cloud data lakes and RDBMS-OLAP systems while reducing duplication, complexity, and operational overhead.
5
The paper decomposes data warehousing into implementation-independent components, capabilities, and practices, then demonstrates how lakehouse architecture satisfies them.

data lakehouse architecture

the capabilities, benefits, and architectural properties of a data lakehouse compared with RDBMS-OLAP systems and cloud data lakes

Publication Details
Publication Date
2023-10-12
Journal
Publisher
ISSN
Cited by
38
Access Type
Author Information
Authors
Dipankar Mazumdar
Hughes, Jason
JB Onofre
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%