On semi-supervised learning

• A. Cholaquidis ^[1] ; R. Fraiman ^[1] ; M. Sued ^[2]
  1. [1] Universidad de la República

     Universidad de la República

     Uruguay

     
  2. [2] Instituto de Cálculo, Facultad de Ciencias Exactas y Naturales, Buenos Aires, Argentina
• Localización: Test: An Official Journal of the Spanish Society of Statistics and Operations Research, ISSN-e 1863-8260, ISSN 1133-0686, Vol. 29, Nº. 4, 2020, págs. 914-937
• Idioma: inglés
• DOI: 10.1007/s11749-019-00690-2
• Texto completo no disponible (Saber más ...)
• Resumen

  • Major efforts have been made, mostly in the machine learning literature, to construct good predictors combining unlabelled and labelled data. These methods are known as semi-supervised. They deal with the problem of how to take advantage, if possible, of a huge amount of unlabelled data to perform classification in situations where there are few labelled data. This is not always feasible: it depends on the possibility to infer the labels from the unlabelled data distribution. Nevertheless, several algorithms have been proposed recently. In this work, we present a new method that, under almost necessary conditions, attains asymptotically the performance of the best theoretical rule when the size of the unlabelled sample goes to infinity, even if the size of the labelled sample remains fixed. Its performance and computational time are assessed through simulations and in the well- known “Isolet” real data of phonemes, where a strong dependence on the choice of the initial training sample is shown. The main focus of this work is to elucidate when and why semi-supervised learning works in the asymptotic regime described above. The set of necessary assumptions, although reasonable, show that semi-parametric methods only attain consistency for very well-conditioned problems.

• Referencias bibliográficas
  • Aaron C, Cholaquidis A, Cuevas A (2017) Stochastic detection of some topological and geometric features. Electron J Stat 11(2):4596–4628....
  • Abdous B, Theodorescu R (1989) On the strong uniform consistency of a new Kernel density estimator. Metrika 11:177–194
  • Agrawala AK (1970) Learning with a probabilistic teacher. IEEE Trans Autom Control 19:716–723
  • Arnold A, Nallapati R, Cohen W (2007) A comparative study of methods for transductive transfer learning. In: Seventh IEEE international conference...
  • Asuncion A, Newman DJ (2007) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences....
  • Azizyan M, Singh A, Wasserman L (2013) Density-sensitive semisupervised inference. Ann Stat 41(2):751–771
  • Belkin M, Niyogi P (2004) Semi-supervised learning on Riemannian manifolds. Mach Learn 56:209–239
  • Ben-David S, Lu T, Pal D (2008) Does unlabelled data provably help? Worst-case analysis of the sample complexity of semi-supervised learning....
  • Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge
  • Chapelle O, Zien A (2005) Semi-supervised classification by low density separation AISTATS, vol 2005, pp 57–64
  • Cholaquidis A, Cuevas A, Fraiman R (2014) On Poincaré cone property. Ann Stat 42:255–284
  • Castelli V, Cover TM (1995) On the exponential value of labeled samples. Pattern Recognit Lett 16(1):105–111
  • Castelli V, Cover TM (1996) The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter. IEEE...
  • Cuevas A, Fraiman R (1997) A plug-in approach to support estimation. Ann Stat 25:2300–2312
  • Cuevas A, Rodríguez-Casal A (2004) On boundary estimation. Adv Appl Probab 36:340–354
  • Cuevas A, Fraiman R, Pateiro-López B (2012) On statistical properties of sets fulfilling rolling-type conditions. Adv Appl Probab 44:311–329
  • Erdös P (1945) Some remarks on the measurability of certain sets. Bull Am Math Soc 51:728–731
  • Fanty M, Cole R (1991) Spoken letter recognition. In: Lippman RP, Moody J, Touretzky DS (eds) Advances in neural information processing systems,...
  • Federer H (1959) Curvature measures. Trans Am Math Soc 93:418–491
  • Fralick SC (1967) Learning to recognize patterns without a teacher. IEEE Trans Inf Theory 13:57–64
  • Haffari G, Sarkar A (2007) Analysis of semi-supervised learning with the Yarowsky algorithm. In: Proceedings of the 23rd conference on uncertainty...
  • Joachims T (1999) Transductive inference for text classification using support vector machines. In: ICML 16
  • Joachims T (2003) Transductive learning via spectral graph partitioning. In: ICML
  • Lafferty J, Wasserman L (2008) Statistical analysis of semi-supervised regression. In: Conference in advances in neural information processing...
  • Nadler B, Srebro N, Zhou X (2009) Statistical analysis of semi-supervised learning: the limit of infinite unlabelled data. In: Bengio Y, Schuurmans...
  • Niyogi P (2008) Manifold regularization and semi-supervised learning: some theoretical analyses. Technical Report TR-2008-01, Computer Science...
  • Rigollet P (2007) Generalized error bound in semi-supervised classification under the cluster assumption. J Mach Learn Res 8:1369–1392 MR2332435
  • Scudder HJ (1965) Probability of error of some adaptive patter-recognition machines. IEEE Trans Inf Theory 11:363–371
  • Singh A, Nowak RD, Zhu X (2008) Unlabeled data: now it helps, now it doesn’t. Technical report, ECE Dept., Univ. Wisconsin-Madison. Available...
  • Sinha K, Belkin M (2009) Semi-supervised learning using sparse eigenfunction bases. In: Bengio Y, Schuurmans D, Lafferty J, Williams CKI,...
  • Vapnik V (1998) Statistical learning theory. Wiley, Hoboken
  • Wang J, Shen X, Pan W (2007) On transductive support vector machines. Contemp Math 443:7–20
  • Zhu X (2008) Semi-supervised learning literature survey. http://pages.cs.wisc.edu/~jerryzhu/research/ssl/semireview.html