Progressive Forest: An Early Stopping Criteria for Building Ensembles

• Autores: Nayma Cepero Pérez, Mailyn Moreno Espino , Milton García Borroto, Eduardo F. Morales
• Localización: Computación y Sistemas (CyS), ISSN 1405-5546, ISSN-e 2007-9737, Vol. 27, Nº. 1, 2023, págs. 89-97
• Idioma: inglés
• DOI: 10.13053/cys-27-1-4224
• Enlaces
  • Texto completo
• Resumen

  • Abstract: Decision forests improve their predictive power based on the combination of various decision trees. The number of trees to be used to achieve the best possible accuracy is not preset and has to be determined by a trial and error process. In many classification problems more trees are used than necessary. This paper introduces a new method, called Progressive Forest, that progressively evaluates the addition of new decision trees into a decision forest to decide when adding more trees is not longer useful. This method was incorporated into the construction schemes of Proactive Forest and Random Forest with very encouraging results. It is experimentally shown that Progressive Forest reduces the number of trees while maintaining the accuracy of the classification. Progressive Forest can be incorporated into any scheme of construction of ensemble, which presents similar characteristics to Random Forest.

• Referencias bibliográficas
  • Banfield, R. E.,Hall, L. O.,Bowyer, K. W.,Kegelmeyer, W. P.. (2006). A comparison of decision tree ensemble creation techniques. IEEE transactions...
  • Breiman, L.. (1996). Bagging predictors. Machine learning. 24. 123
  • Breiman, L.. (2001). Random forests. Machine learning. 45. 5-32
  • Cepero-Pérez, N.,Denis-Miranda, L. A.,Hernández-Palacio, R.,Moreno-Espino, M.,García-Borroto, M.. (2018). Proactive forest for supervised...
  • Derrac, J.,García, S.,Molina, D.,Herrera, F.. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology...
  • Dheeru, D.,Karra-Taniskidou, E.. (2017). UCI machine learning repository.
  • Dietterich, T. G.. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and...
  • Estrada, A.,Morales, E. F.. (2004). Nsc: A new progressive sampling algorithm. workshop on Machine Learning for Scienific data Analysis IBERAMIA....
  • Fan, P.. (2022). Random forest algorithm based on speech for early identification of Parkinson’s disease. Computational Intelligence and Neuroscience....
  • Hernández-Lobato, D.,Martínez-Muñoz, G.,Suárez, A.. (2013). How large should ensembles of classifiers be. Pattern Recognition. 46. 1323
  • Lior, R.. (2019). Ensemble Learning: Pattern Classification Using Ensemble Methods.
  • Lopes, M. E.. (2019). Estimating a sharp convergence bound for randomized ensembles. Journal of Statistical Planning and Inference. 204. 35-44
  • Pereira, D. G.,Afonso, A.,Medeiros, F. M.. (2015). Overview of Friedmans test and post-hoc analysis. Communications in Statistics-Simulation...
  • Provost, F.,Jensen, D.,Oates, T.. (1999). Efficient progressive sampling. fifth ACM SIGKDD international conference on Knowledge discovery...
  • Roka Lior, R.. (2016). Decision forest: Twenty years of research. Information Fusion. 27. 111
  • Sagi, O.,Rokach, L.. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 8. 1249
  • Wang, X.,Chen, A.,Liu, Y.. (2022). Explainable ensemble learning model for predicting steel section-concrete bond strength. Construction and...