Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-02-04T10:24:55.315Z Has data issue: false hasContentIssue false
  • English
  • Français

Sparse Versus Simple Structure Loadings

Published online by Cambridge University Press:  01 January 2025

Nickolay T. Trendafilov  and
Kohei Adachi
Nickolay T. Trendafilov*
Affiliation:
Open University
Kohei Adachi
Affiliation:
Osaka University
*
Correspondence should be sent to Nickolay T. Trendafilov, Department of Mathematics and Statistics, Open University, Walton Hall, Milton Keynes MK7 6AA, UK. Email: Nickolay.Trendafilov@open.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

The component loadings are interpreted by considering their magnitudes, which indicates how strongly each of the original variables relates to the corresponding principal component. The usual ad hoc practice in the interpretation process is to ignore the variables with small absolute loadings or set to zero loadings smaller than some threshold value. This, in fact, makes the component loadings sparse in an artificial and a subjective way. We propose a new alternative approach, which produces sparse loadings in an optimal way. The introduced approach is illustrated on two well-known data sets and compared to the existing rotation methods.

Keywords

principal component analysisfactor analysisorthogonal and oblique rotationssparseness-inducing constraintsLASSO constraintsprojected gradients
Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 3 , September 2015 , pp. 776 - 790
Copyright
Copyright © 2014 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Absil, P-A, Mahony, R., & Sepulchre, R. (2008). Optimization algorithms on matrix manifolds. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
Browne, M.W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150.CrossRefGoogle Scholar
Cadima, J., & Jolliffe, I.T. (1995). Loadings and correlations in the interpretations of principal components. Journal of Applied Statistics, 22, 203–214.CrossRefGoogle Scholar
Candès, E.J., & Tao, T. (2007). The Dantzig selector: Statistical estimation when p\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p$$\end{document} is much larger than n\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n$$\end{document} . Annals of Statistics. 35, 2313–2351.Google Scholar
Harman, H.H. (1976). Modern factor analysis (3rd ed.). Chicago, I: University of Chicago Press.Google Scholar
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24 417–441498–520.CrossRefGoogle Scholar
Jennrich, R.I. (2001). A simple general procedure for orthogonal rotation. Psychometrika, 66, 289–309.CrossRefGoogle Scholar
Jennrich, R.I. (2004). Rotation to simple loadings using component loss functions: The orthogonal case. Psychometrika, 69, 257–273.CrossRefGoogle Scholar
Jennrich, R.I. (2006). Rotation to simple loadings using component loss functions: The oblique case. Psychometrika, 71, 173–191.CrossRefGoogle Scholar
Jennrich, R.I. (2007). Rotation methods, algorithms, and standard errors. In Cudeck, R., & MacCallum, R.C. (Eds.), Factor analysis at 100 (pp. 315–335). Mahwah, NJ: Lawrens Erlbaum Associates.Google Scholar
Jolliffe, I.T. (2002). Principal component analysis (2nd ed.). New York: Springer.Google Scholar
Jolliffe, I.T., Trendafilov, N.T., & Uddin, M. (2003). A modified principal component technique based on the LASSO. Journal of Computational and Graphical Statistics, 12, 531–547.CrossRefGoogle Scholar
Mackey, L., (2009). Deflation methods for sparse PCA. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.), Advances in neural information processing systems, 21, 1017–1024.Google Scholar
MATLAB. (2011). MATLAB R2011a. New York: The MathWorks Inc.Google Scholar
Mulaik, S.A. (2010). The foundations of factor analysis (2nd ed.). Boca Raton, FL: Chapman and Hall/CRC.Google Scholar
Shen, H., & Huang, J.Z. (2008). Sparse principal component analysis via regularized low-rank matrix approximation. Journal of Multivariate Analysis, 99, 1015–1034.CrossRefGoogle Scholar
Thurstone, L.L. (1935). The vectors of mind. Chicago, IL: University of Chicago Press.Google Scholar
Thurstone, L.L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago Press.Google Scholar
Trendafilov, N.T. (2006). The dynamical system approach to multivariate data analysis, a review. Journal of Computational and Graphical Statistics, 50, 628–650.CrossRefGoogle Scholar
Trendafilov, N. T., (2013). From simple structure to sparse components: A review. Computational Statistics. doi:10.1007/s00180-013-0434-5. Special Issue: Sparse Methods in Data Analysis.CrossRefGoogle Scholar
Zou, H., Hastie, T., & Tibshirani, R. (2006). Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15, 265–286.CrossRefGoogle Scholar