Abstract
We present a CUDA-based implementation of a decision tree construction algorithm within the gradient boosting library XGBoost. The tree construction algorithm is executed entirely on the graphics processing unit (GPU) and shows high performance with a variety of datasets and settings, including sparse input matrices. Individual boosting iterations are parallelised, combining two approaches. An interleaved approach is used for shallow trees, switching to a more conventional radix sort-based approach for larger depths. We show speedups of between 3× and 6× using a Titan X compared to a 4 core i7 CPU, and 1.2× using a Titan X compared to 2× Xeon CPUs (24 cores). We show that it is possible to process the Higgs dataset (10 million instances, 28 features) entirely within GPU memory. The algorithm is made available as a plug-in within the XGBoost library and fully supports all XGBoost features including classification, regression and ranking tasks.
Keywords
CUDAComputer scienceXeonTitan (rocket family)Parallel computingGraphics processing unitBoosting (machine learning)General-purpose computing on graphics processing unitsXeon PhiAlgorithmsortSpeedupDecision treeGradient boostingMulti-core processorGraphicsArtificial intelligenceComputer graphics (images)Random forestDatabase
Affiliated Institutions
Related Publications
GPU-acceleration for Large-scale Tree Boosting
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step...
Evaluating the use of GPUs in liver image segmentation and HMMER database searches
In this paper we present the results of parallelizing two life sciences applications, Markov random fields-based (MRF) liver segmentation and HMMER's Viterbi algorithm, using GP...
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
Gradient Boosting Decision Tree (GBDT) is a popular machine learning algorithm, and has quite a few effective implementations such as XGBoost and pGBRT. Although many engineerin...
Neural GPUs Learn Algorithms
Abstract: Learning an algorithm from examples is a fundamental problem that has been widely studied. Recently it has been addressed using neural networks, in particular by Neura...
In-Datacenter Performance Analysis of a Tensor Processing Unit
Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Proc...
Publication Info
- Year
- 2017
- Type
- article
- Volume
- 3
- Pages
- e127-e127
- Citations
- 304
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
304
OpenAlex
Cite This
Rory Mitchell,
Eibe Frank
(2017).
Accelerating the XGBoost algorithm using GPU computing.
PeerJ Computer Science
, 3
, e127-e127.
https://doi.org/10.7717/peerj-cs.127
Identifiers
- DOI
- 10.7717/peerj-cs.127