XGBoost

Tianqi Chen; Carlos Guestrin

doi:10.1145/2939672.2939785

XGBoost

Tianqi Chen, Carlos Guestrin

2016-08-08

SCID: 54.1/uu8j45pm Discuss with AI

XGBoostscalable machine learningsparsity-aware algorithmtree boostingweighted quantile sketch

Abstract (AI)

Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.

Key Findings

A weighted quantile sketch enables approximate tree learning within the XGBoost framework.

By combining algorithmic and systems innovations, XGBoost scales beyond billions of examples while using substantially fewer resources than existing systems.

System-level optimizations involving cache access patterns, data compression, and data sharding improve scalability and resource efficiency.

The system introduces a sparsity-aware algorithm specifically designed to efficiently process sparse data.

XGBoost is presented as a scalable end-to-end tree boosting system that achieves state-of-the-art results across many machine learning challenges.

Research Object

XGBoost end-to-end scalable tree boosting system

Research Subject

scalability and resource-efficient learning performance on sparse and massive datasets

Publication Details

DOI

10.1145/2939672.2939785

Publication Date

2016-08-08

Journal

Publisher

ISSN

Open access PDF

http://dl.acm.org/ft_gateway.cfm?id=2939785&type=pdf

Access Type

Author Information

Authors

Tianqi Chen

Carlos Guestrin

Download PDF

Make a presentation

References available in scid.ai3

Scikit-learn: Machine Learning in Python2011 Greedy function approximation: A gradient boosting machine.2001 Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)2000

Cited by20

AI applications to medical images: From machine learning to deep learning2021 Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances2021 Graph Neural Networks for Intelligent Transportation Systems: A Survey2023 Retrieval augmented generation for large language models in healthcare: A systematic review2025 Machine learning approaches for estimating interfacial tension between oil/gas and oil/water systems: a performance analysis2024 Predicting Spray Dried Dispersion Particle Size Via Machine Learning Regression Methods2022 Integrating permutation feature importance with conformal prediction for robust Explainable Artificial Intelligence in predictive process monitoring2025 Bottomhole Pressure Prediction of Carbonate Reservoirs Using XGBoost2024 A distributional assessment of rivalry in word formation2023 Skor-Xg: Skeleton-Oriented Expected Goal Estimation in Soccer2025 Highly fragmented European wetlands with uneven restoration needs2026 Information Complexity of Time-Frequency Distributions of Signals in Detection and Classification Problems2025 Prediction of Transient NOx Emissions from a Non-Road Heavy-Duty Diesel Engine Based on the PSO-XGBoost Algorithm2026 Schema Matching Using Federated Learning on a Hybrid Feature Set2025 CatBoost for big data: an interdisciplinary review2020 COVID-19 Coronavirus Vaccine Design Using Reverse Vaccinology and Machine Learning2020 The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation2020 A survey on semi-supervised learning2019 Designing Theory-Driven User-Centric Explainable AI2019 On using eXtreme Gradient Boosting (XGBoost) Machine Learning algorithm for Home Network Traffic Classification2019