Decision treesEnsemble learningHandwritten digit recognitionRandom decision forestsRandom subspace method
Figures from the paper
Abstract (AI)
Decision trees are attractive classifiers due to their high execution speed. But trees derived with traditional methods often cannot be grown to arbitrary complexity for possible loss of generalization accuracy on unseen data. The limitation on complexity usually means suboptimal accuracy on training data. Following the principles of stochastic modeling, we propose a method to construct tree-based classifiers whose capacity can be arbitrarily expanded for increases in accuracy for both training and unseen data. The essence of the method is to build multiple trees in randomly selected subspaces of the feature space. Trees in, different subspaces generalize their classification in complementary ways, and their combined classification can be monotonically improved. The validity of the method is demonstrated through experiments on the recognition of handwritten digits.
Key Findings
1
The method's validity is demonstrated empirically on handwritten digit recognition, showing improved accuracy for both training and unseen data.
2
The paper proposes constructing multiple decision trees in randomly selected subspaces of the feature space to allow arbitrarily expandable model capacity.
3
Traditional decision trees often cannot be grown arbitrarily complex without losing generalization accuracy on unseen data, limiting training accuracy.
4
Trees built in different random subspaces generalize complementarily, and combining them can monotonically improve classification performance.
Research Object
Ensemble of decision trees constructed by building multiple trees in randomly selected subspaces of the feature space (random decision forests)
Research Subject
Improving classification accuracy and generalization by expanding model capacity via random subspace tree construction and combining complementary tree classifiers, demonstrated on handwritten digit recognition
Publication Details
Publication Date
2002-11-19
Journal
Publisher
ISSN
Cited by
6271
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by16
Pattern Recognition and Neural Networks1996
A comprehensive survey on machine learning for networking: evolution, applications and research opportunities2018
Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches2019
Self-Driving Laboratories for Chemistry and Materials Science2024
PrankWeb: a web server for ligand binding site prediction and visualization2019
Human emotion recognition from EEG-based brain–computer interface using machine learning: a comprehensive review2022
FlowPrint: Semi-Supervised Mobile-App Fingerprinting on Encrypted Network Traffic2020
Machine Learning Seismic Wave Discrimination: Application to Earthquake Early Warning2018
A comprehensive review of model compression techniques in machine learning2024
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis2024
Artificial intelligence in reproductive medicine2019
Applications of machine learning in metabolomics: Disease modeling and classification2022
Predicting Employee Attrition using Machine Learning2018
How much is the view from the window worth? Machine learning-driven hedonic pricing model of the real estate market2022
An Empirical Evaluation of the t-SNE Algorithm for Data Visualization in Structural Engineering2021
Machine-learning–based knowledge discovery in rheumatoid arthritis–related registry data to identify predictors of persistent pain2019