BigLake: BigQuery's Evolution toward a Multi-Cloud Lakehouse

BigLake: эволюция BigQuery в направлении мультиоблачного lakehouse
Justin J. Levandoski, Garrett Casto, M. Deng, Rushabh Desai, Pavan Edara, Thibaud Hottelier, Amir H. Hormati, Anoop Johnson, Jeff Johnson, Dawid Kurzyniec, Sam McVeety, Prem Ramanathan, Gaurav Saxena, Vidya Shanmugan, Yuri Volobuev
2024-05-23

ACID transactionsBigLakeBigQuerydata lakehousemulti-cloud lakehouse
BigQuery's cloud-native disaggregated architecture has allowed Google Cloud to evolve the system to meet several customer needs across the analytics and AI/ML workload spectrum. A key customer requirement for BigQuery centers around the unification of data lake and enterprise data warehousing workloads. This approach combines: (1) the need for core data management primitives, e.g., security, governance, common runtime metadata, performance acceleration, ACID transactions, provided by an enterprise data warehouses coupled with (2) harnessing the flexibility of the open source format and analytics ecosystem along with new workload types such as AI/ML over unstructured data on object storage. In addition, there is a strong requirement to support BigQuery as a multi-cloud offering given cloud customers are opting for a multi-cloud footprint by default.
1
BigLake addresses customers’ strong requirement to use BigQuery across multi-cloud environments as multi-cloud adoption becomes standard.
2
BigQuery is evolving toward a multi-cloud lakehouse to unify data lake and enterprise data warehouse workloads.
3
The lakehouse approach is intended to support emerging workloads such as AI/ML on unstructured data stored in object storage.
4
The target architecture combines enterprise warehouse capabilities—including security, governance, shared runtime metadata, performance acceleration, and ACID transactions—with open-source data formats and analytics ecosystems.

BigLake, BigQuery’s multi-cloud lakehouse system for unified data lake and enterprise data warehouse workloads

The integration of enterprise data management capabilities with open-format, object-storage-based analytics and AI/ML across multi-cloud environments

Publication Details
Publication Date
2024-05-23
Journal
Publisher
ISSN
Cited by
38
Access Type
Author Information
Authors
Justin J. Levandoski
Garrett Casto
M. Deng
Rushabh Desai
Pavan Edara
Thibaud Hottelier
Amir H. Hormati
Anoop Johnson
Jeff Johnson
Dawid Kurzyniec
Sam McVeety
Prem Ramanathan
Gaurav Saxena
Vidya Shanmugan
Yuri Volobuev
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%