Abstract
This paper deals with a method, called locally linear embedding (LLE). It is a nonlinear dimensionality reduction technique that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional data and attempts to discover a nonlinear structure (including manifolds) in high-dimensional data. In practice, the nonlinear manifold learning methods are applied in image processing, text mining, etc. The implementation of the LLE algorithm is fairly straightforward, because the algorithm has only two control parameters: the number of neighbors of each data point and the regularization parameter. The mapping quality is quite sensitive to these parameters. In this paper, we propose a new way of selecting a regularization parameter of a local Gram matrix.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396
Cristianini N, Taylor JS (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
DeCoste D (2001) Visualizing Mercer kernel feature spaces via kernelized locally linear embeddings. In: Proceedings of the eighth international conference on neural information processing, Shanghai, China, 14–18 November 2001
Donoho DL, Grimes C (2005) Hessian eigenmaps: New locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci 102(21):7426–7431
Dzemyda G (2001) Visualization of a set of parameters characterized by their correlation matrix. Comput Stat Data Anal 36(10):15–30
Gantmacher FR (1988) The theory of matrices. Nauka, Moscow (in Russian)
Ge S, Yang Y, Lee TH (2006) Hand gesture recognition and tracking based on distributed locally linear embedding. In: Proceedings of IEEE conference on robotics, automation and mechatronics, pp 1–6, ISBN:1-4244-0025-2
Hadid A, Kouropteva O, Pietikäinen M (2002) Unsupervised learning using locally linear embedding: experiments with face pose analysis. ICPR (1):111–114
Jain V, Saul LK (2004) Exploratory analysis and visualization of speech and music by locally linear embedding. In: Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP No 29), vol 5, pp 984–987, ISBN:0-7803-8484-9
Karbauskaitė R, Kurasova O, Dzemyda G (2007) Selection of the number of neighbours of each data point for the locally linear embedding algorithm. Inf Technol Control 36(4):359–364. ISSN 1392-124X. Kaunas: Technologija
Kouropteva O, Okun O, Pietikainen M (2002) Selection of the optimal parameter value for the locally linear embedding algorithm. In: Proceedings of 2002 international conference on fuzzy systems and knowledge discovery, pp 359–363
Li HG, Li XG (2004) Gait analysis using LLE. In: Proceedings of the 7th international conference on signal processing: (ICSP’04), vol 3, pp 1423–1426. ISBN:0-7803-8406-7
Li HG, Shi CP, Li XG (2005) LLE based gait recognition. In: Proceedings of 2005 international conference on machine learning and cybernetics, vol 7, pp 4516–4521. ISBN:0-7803-9091-1
Liou CY, Kuo YT (2002) Economic states on neuronic maps. In: Proceedings of the 9th international conference on neural information processing (ICONIP’02), vol 2, pp 787–791
Liu K, Weissenfeld A, Ostermann J (2006) Parameterization of mouth images by LLE and PCA for image-based facial animation. In: ICASSP06, Toulouse, France, IEEE proceedings, vol 5, pp 461–464
Mekuz N, Bauckhage C, Tsotsos JK (2005) Face recognition with weighted locally linear embedding. In: Proceedings of the 2nd Canadian conference on computer and robot vision, pp 290–296. ISBN:0-7695-2319-6
Ridder DD, Kouropteva O, Okun O, Pietikinen M, Duin RPW (2003) Supervized locally linear embedding. Comput Sci 2714:333–341
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326
Saul LK, Roweis ST (2003) Think globally, fit locally: Unsupervised learning of low dimensional manifolds. J Mach Learn Res 4:119–155
Siegel S, Castellan NJ (1988) Nonparametric statistics for the behavioral sciences, 2nd edn. McGraw-Hill, New York
Taylan P, Weber GW, Yerlikaya F (2010) A new approach to multivariate adaptive regression spline by using tikhonov regularization and continuous optimization. TOP Off J Spanish Soc Stat Oper Res
Tenenbaum JB, Silva V Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290:2319–2323
Tikhonov AN, Arsenin AV (1977) Solution of Ill-posed problems. Winston & Sons, Washington. ISBN: 0-470-99124-0
Varini C, Nattkemper TW, Degenhard A, Wismuller A (2004) Breast MRI data analysis by LLE. In: Proceedings of 2004 IEEE international joint conference on neural networks, vol 3, pp 2449–2454. ISBN:0-7803-8359-1
Zhang Z, Zha H (2004) Principal manifolds and nonlinear dimensionality reduction via local tangent space alignment. SIAM J Sci Comput 26(1):313–338
Zhao Q, Zhang D, Lu H (2005) Supervised LLE in ICA space for facial expression recognition. In: Proceedings of international conference on neural networks and brain, ICNN&B ’05, vol 3, pp 1970–1975. ISBN:0-7803-9422-4
Zhu L, Zhu SA (2006) Face recognition based on extended locally linear embedding. In: Proceedings of 2006 1st IEEE conference on industrial electronics and applications, pp 1–4. ISBN:0-7803-9514-X
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Karbauskaitė, R., Dzemyda, G. & Marcinkevičius, V. Dependence of locally linear embedding on the regularization parameter. TOP 18, 354–376 (2010). https://doi.org/10.1007/s11750-010-0151-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11750-010-0151-y
Keywords
- Locally linear embedding
- Dimensionality reduction
- Manifold learning
- High-dimensional data visualization