The Lakehouse: State of the Art on Concepts and Technologies

Lakehouse: современное состояние концепций и технологий
Bernhard Mitschang, J. G. Schneider, Christoph Gröger, Arnold Lutsch, Holger Schwarz
2024-04-18

data lakehousedata lakesdata platformsdata warehousesenterprise analytics
Abstract In the context of data analytics, so-called lakehouses refer to novel variants of data platforms that attempt to combine characteristics of data warehouses and data lakes. In this way, lakehouses promise to simplify enterprise analytics architectures, which often suffer from high operational costs, slow analytical processes and further shortcomings resulting from data replication. However, different views and notions on the lakehouse paradigm exist, which are commonly driven by individual technologies and varying analytical use cases. Therefore, it remains unclear what challenges lakehouses address, how they can be characterized and which technologies can be leveraged to implement them. This paper addresses these issues by providing an extensive overview of concepts and technologies that are related to the lakehouse paradigm and by outlining lakehouses as a distinct architectural approach for data platforms. Concepts and technologies from literature with regard to lakehouses are discussed, based on which a conceptual foundation for lakehouses is established. In addition, several popular technologies are evaluated regarding their suitability for the building of lakehouses. All findings are supported and demonstrated with the help of a representative analytics scenario. Typical challenges of conventional data platforms are identified, a new, sharper definition for lakehouses is proposed and technical requirements for lakehouses are derived. As part of an evaluation, these requirements are applied to several popular technologies, of which frameworks for data lakes turn out to be particularly helpful for the construction of lakehouses. Our work provides an overview of the state of the art and a conceptual foundation for the lakehouse paradigm, which can support future research.
1
An evaluation of popular technologies finds data-lake frameworks particularly suitable for constructing lakehouse architectures.
2
It proposes a sharper lakehouse definition, establishes a conceptual foundation, and derives technical requirements for implementing lakehouses.
3
Lakehouses are characterized as a distinct data-platform architecture combining data warehouse and data lake capabilities to simplify enterprise analytics.
4
The paper identifies conventional platform challenges including high operational costs, slow analytical processing, and shortcomings caused by data replication.
5
The study provides a representative analytics scenario supporting its overview of lakehouse concepts, technologies, challenges, and requirements.

lakehouse data platforms as a distinct architectural approach combining data warehouse and data lake characteristics

the concepts, technical requirements, implementation technologies, and suitability of technologies for building lakehouses

Publication Details
Publication Date
2024-04-18
Journal
Publisher
ISSN
Cited by
71
Access Type
Author Information
Authors
Bernhard Mitschang
J. G. Schneider
Christoph Gröger
Arnold Lutsch
Holger Schwarz
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%