GeoLake: Bringing Geospatial Support to Lakehouses
GeoLake: обеспечение поддержки геопространственных данных в озёрах данных
2023-01-01
SCID: 54.1/ufvyuuw4
Discuss with AI
Lakehouse systemsParquet file formatSpatial SQLgeospatial dataspatial predicate pushdown
Figures from the paper
Abstract (AI)
Over the past decade, geospatial data has witnessed significant expansion, coupled with swift technological advancements. However, its adoption in various industries has been partially constrained by the absence of an efficient data system that can keep pace with these advancements. Among potential solutions, the Lakehouse approach emerges as the most promising. However, existing Lakehouse systems lack integrated support for geospatial data. To address this gap, we present GeoLake, the industry’s first universal solution for geospatial data tailored to Lakehouse systems. GeoLake introduces a range of pivotal features, including native support for storing geometry values in Lakehouse tables, and enables seamless querying and processing of geospatial tables using Spatial SQL. Furthermore, GeoLake incorporates advanced techniques such as spatial partitioning and spatial predicate pushdown within the Lakehouse architecture. These techniques expedite geospatial queries through efficient data skipping and optimized data layout. Additionally, GeoLake incorporates a thoughtfully designed Parquet file format that is tailored for the rapid scanning of encoded geometry values. Our experiments with real-world, large-scale geospatial datasets demonstrate the significant performance improvements achieved by GeoLake. Moreover, several industries, including automotive manufacturing, automobile insurance, and smart city initiatives, some of which are Fortune 500 enterprises, are actively adopting or considering the implementation of GeoLake to address the complexities associated with managing and analyzing geospatial data at scale.
Key Findings
1
A specialized Parquet format enables rapid scanning of encoded geometry values.
2
Experiments on real-world, large-scale geospatial datasets demonstrate significant performance improvements, with adoption or consideration reported across automotive, insurance, and smart-city industries.
3
GeoLake is presented as the first universal geospatial data solution specifically designed for Lakehouse systems.
4
Spatial partitioning and spatial predicate pushdown improve query performance through data skipping and optimized data layout.
5
The system natively stores geometry values in Lakehouse tables and supports seamless querying and processing through Spatial SQL.
Research Object
geospatial data management and analytics in Lakehouse systems
Research Subject
integrated geospatial storage, querying, processing, and query-performance optimization in Lakehouse architectures
Publication Details
Publication Date
2023-01-01
Journal
Publisher
ISSN
Cited by
14
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest