XGBoostscalable machine learningsparsity-aware algorithmtree boostingweighted quantile sketch
Figures from the paper
Abstract (AI)
Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.
Key Findings
1
A weighted quantile sketch enables approximate tree learning within the XGBoost framework.
2
By combining algorithmic and systems innovations, XGBoost scales beyond billions of examples while using substantially fewer resources than existing systems.
3
System-level optimizations involving cache access patterns, data compression, and data sharding improve scalability and resource efficiency.
4
The system introduces a sparsity-aware algorithm specifically designed to efficiently process sparse data.
5
XGBoost is presented as a scalable end-to-end tree boosting system that achieves state-of-the-art results across many machine learning challenges.
Research Object
XGBoost end-to-end scalable tree boosting system
Research Subject
scalability and resource-efficient learning performance on sparse and massive datasets
Publication Details
Publication Date
2016-08-08
Journal
Publisher
ISSN
Cited by
61163
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai3
Cited by20
The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation2020
SoilGrids250m: Global gridded soil information based on machine learning2017
A survey on semi-supervised learning2019
CatBoost for big data: an interdisciplinary review2020
The rise of deep learning in drug discovery2018
Automated Machine Learning2019
Identification of mobile genetic elements with geNomad2023
Anomaly detection in time series: a comprehensive evaluation2022
Designing Theory-Driven User-Centric Explainable AI2019
Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey2019
Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances2021
AI applications to medical images: From machine learning to deep learning2021
Effective Intrusion Detection System Using XGBoost2018
COVID-19 Coronavirus Vaccine Design Using Reverse Vaccinology and Machine Learning2020
Mapping the global depth to bedrock for land surface modeling2016
Graph Neural Networks for Intelligent Transportation Systems: A Survey2023
Machine Learning and Deep Learning in Energy Systems: A Review2022
A Survey on Explainable Anomaly Detection2023
Developing an online hate classifier for multiple social media platforms2020
A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges2025