Statistical significance of clustering for high-dimension, low-sample size data

• Autores: Yufeng Liu, David Haynes, Andrew Nobel, J. S. Marron
• Localización: Journal of the American Statistical Association, ISSN 0162-1459, Vol. 103, Nº 483, 2008
• Idioma: inglés
• DOI: 10.1198/016214508000000454
• Texto completo no disponible (Saber más ...)
• Resumen

  • Clustering methods provide a powerful tool for the exploratory analysis of high-dimension, low-sample size (HDLSS) data sets, such as gene expression microarray data. A fundamental statistical issue in clustering is which clusters are "really there," as opposed to being artifacts of the natural sampling variation. We propose SigClust as a simple and natural approach to this fundamental statistical problem. In particular, we define a cluster as data coming from a single Gaussian distribution and formulate the problem of assessing statistical significance of clustering as a testing procedure. This Gaussian null assumption allows direct formulation of p values that effectively quantify the significance of a given clustering. HDLSS covariance estimation for SigClust is achieved by a combination of invariance principles, together with a factor analysis model. The properties of SigClust are studied. Simulated examples, as well as an application to a real cancer microarray data set, show that the proposed method works remarkably well for assessing significance of clustering. Some theoretical results also are obtained. [PUBLICATION ABSTRACT]