BigLake: BigQuery's Evolution toward a Multi-Cloud Lakehouse
BigLake: эволюция BigQuery в направлении мультиоблачного lakehouse
2024-05-23
SCID: 54.1/hwfs82jq
Discuss with AI
ACID transactionsBigLakeBigQuerydata lakehousemulti-cloud lakehouse
Figures from the paper
Abstract (AI)
BigQuery's cloud-native disaggregated architecture has allowed Google Cloud to evolve the system to meet several customer needs across the analytics and AI/ML workload spectrum. A key customer requirement for BigQuery centers around the unification of data lake and enterprise data warehousing workloads. This approach combines: (1) the need for core data management primitives, e.g., security, governance, common runtime metadata, performance acceleration, ACID transactions, provided by an enterprise data warehouses coupled with (2) harnessing the flexibility of the open source format and analytics ecosystem along with new workload types such as AI/ML over unstructured data on object storage. In addition, there is a strong requirement to support BigQuery as a multi-cloud offering given cloud customers are opting for a multi-cloud footprint by default.
Key Findings
1
BigLake addresses customers’ strong requirement to use BigQuery across multi-cloud environments as multi-cloud adoption becomes standard.
2
BigQuery is evolving toward a multi-cloud lakehouse to unify data lake and enterprise data warehouse workloads.
3
The lakehouse approach is intended to support emerging workloads such as AI/ML on unstructured data stored in object storage.
4
The target architecture combines enterprise warehouse capabilities—including security, governance, shared runtime metadata, performance acceleration, and ACID transactions—with open-source data formats and analytics ecosystems.
Research Object
BigLake, BigQuery’s multi-cloud lakehouse system for unified data lake and enterprise data warehouse workloads
Research Subject
The integration of enterprise data management capabilities with open-format, object-storage-based analytics and AI/ML across multi-cloud environments
Publication Details
Publication Date
2024-05-23
Journal
Publisher
ISSN
Cited by
38
Access Type
Author Information
Download PDF
Subscribe to digest