Abstract
Different performance measures are used to inspect, compare and evaluate the behaviour of classifiers in Machine Learning (ML). ML researchers employ one or several of these performance measures in their classification studies to report their success. However, no widespread consensus has been reached on a unified chosen measure. In this work, we introduce a reliable and informative measure, the Unified Performance Measure (UPM), by modifying the \(F_1\)-score to avoid its undesired behaviour in imbalanced classification problems. UPM is compared with alternative performance measures, like the \(F_1\)-score or Accuracy, in both simulated confusion matrices and real datasets. The proposed measure outperforms the alternatives, providing a promising new research line.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Canbek, G., Sagiroglu, S., Temizel, T.T., Baykal, N.: Binary classification performance measures/metrics: a comprehensive visualized roadmap to gain new insights. In: 2017 International Conference on Computer Science and Engineering (UBMK), pp. 821–826. IEEE (2017)
Digitanimal (2020). https://www.digitanimal.com/
Goodall, D.W.: The distribution of the matching coefficient. Biometrics 9, 647–656 (1967)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Matthews, B.W.: Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Struct. 405(2), 442–451 (1975)
Muñoz, J., Moguerza, J., Martín Duque, C., Bruna, D.: A study on the effect of imbalanced data in tourism recommendation models. Int. J. Qual. Serv. Sci. ahead-of-print, 08 2019. https://doi.org/10.1108/IJQSS-05-2018-0050
Navarro, J., Diego, I.M.d., Fernández-Isabel, A., Ortega, F.: Fusion of GPS and accelerometer information for anomalous trajectories detection. In: Proceedings of the 2019 the 5th International Conference on e-Society, e-Learning and e-Technologies, pp. 52–57 (2019)
Sasaki, Y., et al.: The truth of the f-measure, vol. 2007 (2007)
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manage. 45(4), 427–437 (2009)
Acknowledgments
This research has been supported by grants from Madrid Autonomous Community (Ref: IND2018/TIC-9665) and the Spanish Science and Innovation, under the Retos-Colaboración program: SABERMED (Ref: RTC-2017-6253-1); and the Retos-Investigación program:MODAS-IN (reference: RTI-2018-094269-B-I00). Special thanks to MISC International S.L and HOTELS QUALITY.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Redondo, A.R., Navarro, J., Fernández, R.R., de Diego, I.M., Moguerza, J.M., Fernández-Muñoz, J.J. (2020). Unified Performance Measure for Binary Classification Problems. In: Analide, C., Novais, P., Camacho, D., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2020. IDEAL 2020. Lecture Notes in Computer Science(), vol 12490. Springer, Cham. https://doi.org/10.1007/978-3-030-62365-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-62365-4_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62364-7
Online ISBN: 978-3-030-62365-4
eBook Packages: Computer ScienceComputer Science (R0)