The HANA Native Query Engine for Lakehouse Systems

Собственный механизм выполнения запросов HANA для систем Lakehouse
P Willems, Daniel Ritter, Norman May, Taehyung Lee, Mihnea Andrei, Sukhyeun Cho, Maik Görgens, Amit Pathak, Paul R. Willems
2025-08-01

Analytical query processingDirect access architectureHANA Native Query EngineLakehouse systemsSQL-on-Files
Modern enterprise applications and data warehouse systems move data into data lakes for economical and scalability reasons. Data is then stored in popular columnar file formats like Parquet which are optimized for writing using open table formats like Iceberg or Delta. This presents new challenges for existing database systems and their execution engines because excellent performance and scalability when accessing this data in complex analytical queries is expected while data is located in a remote data lake. In this work, we present how we adapted the HANA Cloud Database Engine for efficient processing of files in data lakes, which we call SQL-on-Files (SoF). We motivate this evolution by its relevance for Business Data Cloud, SAP's Lakehouse, we discuss the viability of general architecture choices like pushdown and direct access architectures, and give insights into our SoF design decisions towards scalable, analytical query processing around execution engine, optimizer and caching. Our evaluation of SoF shows benefits of direct access over pushdown architectures for a new warehouse benchmark with complex, analytical workloads.
1
Evaluation on a new warehouse benchmark with complex analytical workloads shows that direct access outperforms pushdown architectures.
2
SQL-on-Files targets scalable analytical processing for SAP’s Business Data Cloud and lakehouse environments.
3
The HANA Cloud Database Engine was adapted to process Parquet and open-table-format data directly in remote data lakes through a SQL-on-Files architecture.
4
The paper analyzes pushdown and direct-access architectures for lakehouse query processing, including execution engine, optimizer, and caching design choices.

HANA Cloud Database Engine adapted for processing Parquet files in remote data lakes within lakehouse systems

Scalable and efficient execution of complex analytical queries, including the performance of direct-access versus pushdown architectures and related engine, optimizer, and caching design choices

Publication Details
Publication Date
2025-08-01
Journal
Publisher
ISSN
Cited by
7
Access Type
Author Information
Authors
P Willems
Daniel Ritter
Norman May
Taehyung Lee
Mihnea Andrei
Sukhyeun Cho
Maik Görgens
Amit Pathak
Paul R. Willems
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%