Hostname: page-component-76c49bb84f-c7tcl Total loading time: 0 Render date: 2025-07-04T17:21:03.817Z Has data issue: false hasContentIssue false
  • English
  • Français

Sparse linear least-squares problems

Part of: Numerical linear algebra

Published online by Cambridge University Press:  01 July 2025

Jennifer Scott Open the ORCID record for Jennifer Scott [Opens in a new window]  and
Miroslav Tůma
Jennifer Scott
Affiliation:
Department of Mathematics and Statistics, School of Mathematical, Physical and Computational Sciences, University of Reading, Reading RG6 6AQ, UK and Scientific Computing Department, STFC Rutherford Appleton Laboratory, Harwell Campus, Didcot, Oxfordshire, OX11 0QX, UK E-mail: jennifer.scott@reading.ac.uk
Miroslav Tůma
Affiliation:
Department of Numerical Mathematics, Faculty of Mathematics and Physics, Charles University, Sokolovská 49/83, 186 75 Praha 8, Czech Republic E-mail: mirektuma@karlin.mff.cuni.cz
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Least-squares problems are a cornerstone of computational science and engineering. Over the years, the size of the problems that researchers and practitioners face has constantly increased, making it essential that sparsity is exploited in the solution process. The goal of this article is to present a broad review of key algorithms for solving large-scale linear least-squares problems. This includes sparse direct methods and algebraic preconditioners that are used in combination with iterative solvers. Where software is available, this is highlighted.

MSC classification

Primary: 65F20: Overdetermined systems, pseudoinverses 65F50: Sparse matrices
Secondary: 65F05: Direct methods for linear systems and matrix inversion 65F10: Iterative methods for linear systems 65F08: Preconditioners for iterative methods

Information

Type
Research Article
Information
Acta Numerica , Volume 34 , July 2025 , pp. 891 - 1010
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2025. Published by Cambridge University Press

References

Adlers, M. and Björck, Å. (2000), Matrix stretching for sparse least squares problems, Numer . Linear Algebra Appl. 7, 51–65.10.1002/(SICI)1099-1506(200003)7:2<51::AID-NLA187>3.0.CO;2-O3.0.CO;2-O>CrossRefGoogle Scholar
Agullo, E., Buttari, A., Guermouche, A. and Lopez, F. (2016), Implementing multifrontal sparse solvers for multicore architectures with sequential task flow runtime systems, ACM Trans. Math. Softw. 43, art. 13.Google Scholar
Ajiz, M. A. and Jennings, A. (1984), A robust incomplete Choleski-conjugate gradient algorithm, Int. J. Numer. Methods Engrg 20, 949–966.10.1002/nme.1620200511CrossRefGoogle Scholar
Al Daas, H. and Grigori, L. (2019), A class of efficient locally constructed preconditioners based on coarse spaces, SIAM J. Matrix Anal. Appl. 40, 66–91.10.1137/18M1194365CrossRefGoogle Scholar
Al Daas, H., Jolivet, P. and Scott, J. A. (2022), A robust algebraic domain decomposition preconditioner for sparse normal equations, SIAM J. Sci. Comput. 44, A1047–A1068.10.1137/21M1434891CrossRefGoogle Scholar
Amestoy, P. R., Davis, T. A. and Duff, I. S. (1996a), An approximate minimum degree ordering algorithm, SIAM J. Matrix Anal. Appl. 17, 886–905.10.1137/S0895479894278952CrossRefGoogle Scholar
Amestoy, P. R., Davis, T. A. and Duff, I. S. (2004), Algorithm 837: AMD, an approximate minimum degree ordering algorithm, ACM Trans. Math. Softw. 30, 381–388.10.1145/1024074.1024081CrossRefGoogle Scholar
Amestoy, P. R., Duff, I. S. and L’Excellent, J.-Y. (2000), Multifrontal parallel distributed symmetric and unsymmetric solvers, Comput. Methods Appl. Mech. Engrg 184, 501–520.10.1016/S0045-7825(99)00242-XCrossRefGoogle Scholar
Amestoy, P. R., Duff, I. S. and Puglisi, C. (1996b), Multifrontal QR factorization in a multiprocessor environment, Numer . Linear Algebra Appl. 3, 275–300.10.1002/(SICI)1099-1506(199607/08)3:4<275::AID-NLA83>3.0.CO;2-73.0.CO;2-7>CrossRefGoogle Scholar
Arioli, M., Duff, I. S. and de Rijk, P. P. M. (1989), On the augmented system approach to sparse least-squares problems, Numer. Math. 55, 667–684.10.1007/BF01389335CrossRefGoogle Scholar
Arridge, S. R., Betcke, M. M. and Harhanen, L. (2014), Iterated preconditioned LSQR method for inverse problems on unstructured grids, Inverse Problems 30, art. 075009.10.1088/0266-5611/30/7/075009CrossRefGoogle Scholar
Ashcraft, C. (1995), Compressed graphs and the minimum degree algorithm, SIAM J. Sci. Comput. 16, 1404–1411.10.1137/0916081CrossRefGoogle Scholar
Ashcraft, C. and Grimes, R. (1989), The influence of relaxed supernode partitions on the multifrontal method, ACM Trans. Math. Softw. 15, 291–309.10.1145/76909.76910CrossRefGoogle Scholar
Avron, H., E. Ng and Toledo, S. (2009), Using perturbed QR factorizations to solve linear least-squares problems, SIAM J. Matrix Anal. Appl. 31, 674–693.10.1137/070698725CrossRefGoogle Scholar
Baglama, J. and Richmond, D. J. (2014), Implicitly restarting the LSQR algorithm, Electron . Trans. Numer. Anal. 42, 85–105.Google Scholar
Bai, Z.-Z. and Yin, J.-F. (2009), Modified incomplete orthogonal factorization methods using Givens rotations, Computing 86, 53–69.10.1007/s00607-009-0065-4CrossRefGoogle Scholar
Ballard, G., Demmel, J., Grigori, L., Jacquelin, M., Knight, N. and Nguyen, H. D. (2015), Reconstructing Householder vectors from tall-skinny QR, J. Parallel Distrib. Comput. 85, 3–31.10.1016/j.jpdc.2015.06.003CrossRefGoogle Scholar
Barlow, J. L. (2013), Reorthogonalization for the Golub–Kahan–Lanczos bidiagonal reduction, Numer . Math. 124, 237–278.Google Scholar
Barnard, S. T., Pothen, A. and Simon, H. D. (1995), A spectral algorithm for envelope reduction of sparse matrices, Numer . Linear Algebra Appl. 2, 317–334.10.1002/nla.1680020402CrossRefGoogle Scholar
Bartels, R. H.,Golub, G. H.andSaunders, M. A.(1970), Numerical techniques in mathematical programming, in Nonlinear Programming (Proc. Sympos. Univ. Wisconsin–Madison), Academic Press, pp. 123–176.Google Scholar
Bellavia, S., Gondzio, J. and Morini, B. (2013), A matrix-free preconditioner for sparse symmetric positive definite systems and least-squares problems, SIAM J. Sci. Comput. 35, A192–A211.10.1137/110840819CrossRefGoogle Scholar
Benbow, S. J. (1999), Solving generalized least-squares problems with LSQR, SIAM J. Matrix Anal. Appl. 21, 166–177.10.1137/S0895479897321830CrossRefGoogle Scholar
Benoit, C. (1924), Note sur une méthode de résolution des équations normales provenant de l’application de la méthode des moindres carrés à un système d’équations linéaires en nombre inférieur à celui des inconnues: Application de la méthode à la résolution d’un systeme défini d’équations linéaires, Bulletin Géodésique 2, 5–77.Google Scholar
Benson, A. R., Gleich, D. F. and Demmel, J. (2013), Direct QR factorizations for tall-and-skinny matrices in MapReduce architectures, in 2013 IEEE International Conference on Big Data, IEEE, pp. 264–272.Google Scholar
Benzi, M. (2004), Gianfranco Cimmino’s contributions to numerical mathematics. Atti del Seminario di Analisi Matematica dell’Università di Bologna, pp. 87–109.Google Scholar
Benzi, M. and Faccio, C. (2024), Solving linear systems of the form by preconditioned iterative methods, SIAM J. Sci. Comput. 46, S51–S70.10.1137/22M1505529CrossRefGoogle Scholar
Benzi, M. and Tůma, M. (2003a), A robust incomplete factorization preconditioner for positive definite matrices, Numer . Linear Algebra Appl. 10, 385–400.10.1002/nla.320CrossRefGoogle Scholar
Benzi, M. and Tůma, M. (2003b), A robust preconditioner with low memory requirements for large sparse least squares problems, SIAM J. Sci. Comput. 25, 499–512.10.1137/S106482750240649XCrossRefGoogle Scholar
Benzi, M., Golub, G. and Liesen, J. (2005), Numerical solution of saddle point problems, Acta Numer. 14, 1–137.10.1017/S0962492904000212CrossRefGoogle Scholar
Berman, A. and Plemmons, R. J. (1974), Cones and iterative methods for best least squares solutions of linear systems, SIAM J. Numer. Anal. 11, 145–154.10.1137/0711015CrossRefGoogle Scholar
Bichot, C.-E. and Siarry, P., eds (2011), Graph Partitioning, ISTE and Wiley.Google Scholar
Bischof, C. H. and Hansen, P. C. (1991), Structure-preserving and rank-revealing QR-factorizations, SIAM J. Sci. Comput. 12, 1332–1350.10.1137/0912073CrossRefGoogle Scholar
Bischof, C. H. and Hansen, P. C. (1992), A block algorithm for computing rank-revealing QR factorizations, Numer . Algorithms 2, 371–391.10.1007/BF02139475CrossRefGoogle Scholar
Björck, Å. (1967a), Iterative refinement of linear least squares solutions I, BIT Numer. 7, 257–278.10.1007/BF01939321CrossRefGoogle Scholar
Björck, Å. (1967b), Solving linear least squares problems by Gram–Schmidt orthogonalization, BIT Numer. 7, 1–21.10.1007/BF01934122CrossRefGoogle Scholar
Björck, Å. (1976), Methods for sparse least squares problems, in Sparse Matrix Computations (Bunch, J. R. and Rose, D. J., eds), Academic Press, pp. 177–199.10.1016/B978-0-12-141050-6.50015-5CrossRefGoogle Scholar
Björck, Å. (1987), Stability analysis of the method of seminormal equations for linear least squares problems, Linear Algebra Appl. 88, 31–48.10.1016/0024-3795(87)90101-7CrossRefGoogle Scholar
Björck, Å. (2024), Numerical Methods for Least Squares Problems, second edition, SIAM.10.1137/1.9781611977950CrossRefGoogle Scholar
Björck, Å. and Golub, G. (1967), ALGOL programming, contribution no. 22: Iterative refinement of linear least square solutions by Householder transformation, BIT Numer. Math. 7, 322–337.Google Scholar
Björck, Å. and Paige, C. C. (1994), Solution of augmented linear systems using orthogonal factorizations, BIT Numer. Math. 34, 1–24.10.1007/BF01935013CrossRefGoogle Scholar
Braess, D. and Peisker, P. (1986), On the numerical solution of the biharmonic equation and the role of squaring matrices for preconditioning, IMA J. Numer. Anal. 6, 393–404.10.1093/imanum/6.4.393CrossRefGoogle Scholar
Bru, R., Marín, J., Mas, J. and Tůma, M. (2014), Preconditioned iterative methods for solving linear least squares problems, SIAM J. Sci. Comput. 36, A2002–A2022.10.1137/130931588CrossRefGoogle Scholar
Buleev, N. I. (1959), A numerical method for solving two-dimensional diffusion equations (in Russian), Atomnaja Energija 6, 338–340.Google Scholar
Bunch, J. R. and Kaufman, L. (1977), Some stable methods for calculating inertia and solving symmetric linear systems, Math. Comp. 31, 162–179.10.1090/S0025-5718-1977-0428694-0CrossRefGoogle Scholar
Businger, P. and Golub, G. H. (1965), Linear least squares solutions by Householder transformations, Numer. Math. 7, 269–276.10.1007/BF01436084CrossRefGoogle Scholar
Buttari, A. (2013), Fine-grained multithreading for the multifrontal QR factorization of sparse matrices, SIAM J. Sci. Comput. 35, C323–C345.10.1137/110846427CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2005), Priorconditioners for linear systems, Inverse Problems 21, art. 1397.10.1088/0266-5611/21/4/014CrossRefGoogle Scholar
Cambier, L., Chen, C., Boman, E. G., Rajamanickam, S., Tuminaro, R. S. and Darve, E. (2020), An algebraic sparsified nested dissection algorithm using low-rank approximations, SIAM J. Matrix Anal. Appl. 41, 715–746.10.1137/19M123806XCrossRefGoogle Scholar
Carson, E. and Daužickaitė, I. (2024), A comparison of mixed precision iterative refinement approaches for least-squares problems. Available at arXiv:2405.18363.Google Scholar
Carson, E. and Higham, N. J. (2017), A new analysis of iterative refinement and its application to accurate solution of ill-conditioned sparse linear systems, SIAM J. Sci. Comput. 39, A2834–A2856.10.1137/17M1122918CrossRefGoogle Scholar
Carson, E. and Higham, N. J. (2018), Accelerating the solution of linear systems by iterative refinement in three precisions, SIAM J. Sci. Comput. 40, A817–A847.10.1137/17M1140819CrossRefGoogle Scholar
Carson, E., Higham, N. J. and Pranesh, S. (2020), Three-precision GMRES-based iterative refinement for least squares problems, SIAM J. Sci. Comput. 42, A4063–A4083.10.1137/20M1316822CrossRefGoogle Scholar
Carson, E., Lund, K., Rozložnk, M. and Thomas, S. (2022), Block Gram–Schmidt algorithms and their stability properties, Linear Algebra Appl. 638, 150–195.10.1016/j.laa.2021.12.017CrossRefGoogle Scholar
Cartis, C., Fiala, J. and Shao, Z. (2021), Hashing embeddings of optimal dimension, with applications to linear least squares. Available at arXiv:2105.11815.Google Scholar
Çatalyürek, U. V., Aykanat, C. and Kayaaslan, E. (2011), Hypergraph partitioning-based fill-reducing ordering for symmetric matrices, SIAM J. Sci. Comput. 33, 1996–2023.10.1137/090757575CrossRefGoogle Scholar
Cerdán, J., Guerrero, D., Marín, J. and Mas, J. (2020), Preconditioners for rank deficient least squares problems, J. Comput. Appl. Math. 372, art. 112621.10.1016/j.cam.2019.112621CrossRefGoogle Scholar
Chan, T. F. (1987), Rank revealing QR factorizations, Linear Algebra Appl. 88/89, 67–82.Google Scholar
Chan, T. F. and van der Vorst, H. A. (1997), Approximate and incomplete factorizations, in Parallel Numerical Algorithms (D. E. Keyes, A. Sameh and Venkatakrishnan, V., eds), Kluwer Academic, pp. 167–202.10.1007/978-94-011-5412-3_6CrossRefGoogle Scholar
Chandrasekaran, S. and Ipsen, I. C. F. (1994), On rank-revealing factorisations, SIAM J. Matrix Anal. Appl. 15, 592–622.10.1137/S0895479891223781CrossRefGoogle Scholar
Chang, X.-W., Paige, C. C. and Titley-Peloquin, D. (2009), Stopping criteria for the iterative solution of linear least squares problems, SIAM J. Matrix Anal. Appl. 31, 831–852.10.1137/080724071CrossRefGoogle Scholar
Chen, D., Huang, T.-Z. and Li, L. (2012), An algorithm for symmetric indefinite linear systems, J. Comput. Anal. Appl. 14, 767–784.Google Scholar
Chen, Q., Ghai, A. and Jiao, X. (2021), HILUCSI: Simple, robust, and fast multilevel ILU for large-scale saddle-point problems from PDEs, Numer . Linear Algebra Appl. 28, art. e2400.10.1002/nla.2400CrossRefGoogle Scholar
Chen, Y., Davis, T. A., Hager, W. W. and Rajamanickam, S. (2008), Algorithm 887: CHOLMOD, supernodal sparse Cholesky factorization and update/downdate, ACM Trans. Math. Softw. 35, 1–14.10.1145/1391989.1391995CrossRefGoogle Scholar
Çivril, A. and Magdon-Ismail, M. (2009), On selecting a maximum volume sub-matrix of a matrix and related problems, Theoret . Comput. Sci. 410, 4801–4811.Google Scholar
Climent, J.-J. and Perea, C. (2003), Iterative methods for least-square problems based on proper splittings, J. Comput. Appl. Math. 158, 43–48.10.1016/S0377-0427(03)00465-5CrossRefGoogle Scholar
Coleman, T. F. and Verma, A. (2001), A preconditioned conjugate gradient approach to linear equality constrained minimization, Comput. Optim. Appl. 20, 61–72.10.1023/A:1011271406353CrossRefGoogle Scholar
Coleman, T. F., Edenbrandt, A. and Gilbert, J. R. (1986), Predicting fill for sparse orthogonal factorization, J. Assoc. Comput. Mach. 33, 517–532.10.1145/5925.5932CrossRefGoogle Scholar
Constantine, P. G.andGleich, D. F.(2011), Tall and skinny QR factorizations in mapreduce architectures, inProceedings of the Second International Workshop on MapReduce and its Applications (MapReduce ’11), ACM, pp. 43–50.10.1145/1996092.1996103CrossRefGoogle Scholar
Craig, E. J. (1955), The N-step iteration procedures, J. Math. Phys. 34, 64–73.10.1002/sapm195534164CrossRefGoogle Scholar
Cuthill, E. H. and McKee, J. (1969), Reducing the bandwidth of sparse symmetric matrices, in Proceedings of the 24th National Conference of the ACM, ACM Press, pp. 157–172.Google Scholar
Daužickaitė, I., Lawless, A. S., Scott, J. A. and Van Leeuwen, P. J. (2021), Randomised preconditioning for the forcing formulation of weak-constraint 4D-Var, Quart . J. R. Meteorol. Soc. 147, 3719–3734.10.1002/qj.4151CrossRefGoogle Scholar
Davis, T. A. (2011), Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization, ACM Trans. Math. Softw. 38, art. 8.10.1145/2049662.2049670CrossRefGoogle Scholar
Davis, T. A. and Hager, W. W. (2001), Multiple-rank modifications of a sparse Cholesky factorization, SIAM J. Matrix Anal. Appl. 22, 997–1013.10.1137/S0895479899357346CrossRefGoogle Scholar
Davis, T. A., Gilbert, J. R., Larimore, S. I. and Ng, E. G. (2004a), Algorithm 836: COLAMD, a column approximate minimum degree ordering algorithm, ACM Trans. Math. Softw. 30, 377–380.10.1145/1024074.1024080CrossRefGoogle Scholar
Davis, T. A., Gilbert, J. R., Larimore, S. I. and Ng, E. G. (2004b), A column approximate minimum degree ordering algorithm, ACM Trans. Math. Softw. 30, 353–376.10.1145/1024074.1024079CrossRefGoogle Scholar
Davis, T. A., Hager, W. W., Kolodziej, S. P. and Yeralan, S. N. (2020), Algorithm 1003: Mongoose, a graph coarsening and partitioning library, ACM Trans. Math. Softw. 46, art. 7.10.1145/3337792CrossRefGoogle Scholar
Davis, T. A., Rajamanickam, S. and Sid-Lakhdar, W. M. (2016), A survey of direct methods for sparse linear systems, Acta Numer. 25, 383–566.10.1017/S0962492916000076CrossRefGoogle Scholar
de Hoog, F. and Hegland, M. (2023), A note on error bounds for pseudo skeleton approximations of matrices, Linear Algebra Appl. 669, 102–117.10.1016/j.laa.2023.03.024CrossRefGoogle Scholar
Dehghani, M., Lambe, A. and Orban, D. (2020), A regularized interior-point method for constrained linear least squares, Inf . Syst. Oper. Res. 58, 202–224.Google Scholar
Demmel, J., Grigori, L., Hoemmen, M. and Langou, J. (2012), Communication-optimal parallel and sequential QR and LU factorizations, SIAM J. Sci. Comput. 34, A206–A239.10.1137/080731992CrossRefGoogle Scholar
Demmel, J. W. (1997), Applied Numerical Linear Algebra, SIAM.10.1137/1.9781611971446CrossRefGoogle Scholar
Demmel, J. W., Grigori, L., Gu, M. and Xiang, H. (2015), Communication avoiding rank revealing QR factorization with column pivoting, SIAM J. Matrix Anal. Appl. 36, 55–89.10.1137/13092157XCrossRefGoogle Scholar
Demmel, J. W., Hida, Y., Riedy, J. E. and Li, X. S. (2009), Extra-precise iterative refinement for overdetermined least squares problems, ACM Trans. Math. Softw. 35, 1–32.10.1145/1462173.1462177CrossRefGoogle Scholar
di Serafino, D. and Orban, D. (2021), Constraint-preconditioned Krylov solvers for regularized saddle-point systems, SIAM J. Sci. Comput. 43, A1001–A1026.10.1137/19M1291753CrossRefGoogle Scholar
Drineas, P. and Mahoney, M. W. (2016), RandNLA: Randomized numerical linear algebra, Commun . Assoc. Comput. Mach. 59, 80–90.Google Scholar
Duersch, J. A. and Gu, M. (2020), Randomized projection for rank-revealing matrix factorizations and low-rank approximations, SIAM Rev. 62, 661–682.10.1137/20M1335571CrossRefGoogle Scholar
Duff, I., Hogg, J. and Lopez, F. (2020), A new sparse LDL T solver using a posteriori threshold pivoting, SIAM J. Sci. Comput. 42, C23–C42.10.1137/18M1225963CrossRefGoogle Scholar
Duff, I. S. (2004), MA57: A code for the solution of sparse symmetric definite and indefinite systems, ACM Trans. Math. Softw. 30, 118–154.10.1145/992200.992202CrossRefGoogle Scholar
Duff, I. S. and Pralet, S. (2005), Strategies for scaling and pivoting for sparse symmetric indefinite problems, SIAM J. Matrix Anal. Appl. 27, 313–340.10.1137/04061043XCrossRefGoogle Scholar
Duff, I. S. and Pralet, S. (2007), Towards stable mixed pivoting strategies for the sequential and parallel solution of sparse symmetric indefinite systems, SIAM J. Matrix Anal. Appl. 29, 1007–1024.10.1137/050629598CrossRefGoogle Scholar
Duff, I. S. and Reid, J. K. (1983), The multifrontal solution of indefinite sparse symmetric linear, ACM Trans. Math. Softw. 9, 302–325.10.1145/356044.356047CrossRefGoogle Scholar
Duff, I. S., Erisman, A. M. and Reid, J. K. (2017), Direct Methods for Sparse Matrices, second edition, Oxford University Press.10.1093/acprof:oso/9780198508380.001.0001CrossRefGoogle Scholar
Duff, I. S., Gould, N. I. M., Reid, J. K., Scott, J. A. and Turner, K. (1991), The factorization of sparse symmetric indefinite matrices, IMA J. Numer. Anal. 11, 181–204.10.1093/imanum/11.2.181CrossRefGoogle Scholar
Duff, I. S., Guivarch, R., Ruiz, D. and Zenadi, M. (2015), The augmented block Cimmino distributed method, SIAM J. Sci. Comput. 37, A1248–A1269.10.1137/140961444CrossRefGoogle Scholar
Dumitraşc, A., Leleux, P., Popa, C., Ruede, U. and Ruiz, D. (2021), Extensions of the augmented block Cimmino method to the solution of full rank rectangular systems, SIAM J. Sci. Comput. 43, S516–S539.10.1137/20M1348261CrossRefGoogle Scholar
Dumitraşc, A., Leleux, P., Popa, C., Ruiz, D. and Torun, S. (2018), The augmented block Cimmino algorithm revisited. Available at arXiv:1805.11487.Google Scholar
Edlund, O. (2002), A software package for sparse orthogonal factorization and updating, ACM Trans. Math. Softw. 28, 448–482.10.1145/592843.592848CrossRefGoogle Scholar
Elfving, T. (1980), Block-iterative methods for consistent and inconsistent linear equations, Numer . Math. 35, 1–12.Google Scholar
Epperly, E. N. (2024), Fast and forward stable randomized algorithms for linear least-squares problems, SIAM J. Matrix Anal. Appl. 45, 1782–1804.10.1137/23M1616790CrossRefGoogle Scholar
Epperly, E. N., Meier, M. and Nakatsukasa, Y. (2024), Fast randomized least-squares solvers can be just as accurate and stable as classical direct solvers. Available at arXiv:2406.03468.Google Scholar
Estrin, R., Orban, D. and Saunders, M. A. (2019), LSLQ: An iterative method for linear least-squares with an error minimization property, SIAM J. Matrix Anal. Appl. 40, 254–275.10.1137/17M1113552CrossRefGoogle Scholar
Fong, D. C.-L. and Saunders, M. (2011), LSMR: An iterative algorithm for sparse least-squares problems, SIAM J. Sci. Comput. 33, 2950–2971.10.1137/10079687XCrossRefGoogle Scholar
Foster, L. V. (1986), Rank and null space calculations using matrix decomposition without column interchanges, Linear Algebra Appl. 74, 47–71.10.1016/0024-3795(86)90115-1CrossRefGoogle Scholar
Francis, J. G. F. (1961), The QR transformation a unitary analogue to the LR transformation, Part 1, Comput. J. 4, 265–271.10.1093/comjnl/4.3.265CrossRefGoogle Scholar
Freund, R. W. (1997), Preconditioning of symmetric, but highly indefinite linear systems, in 15th IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics, Vol. 2, pp. 551–556.Google Scholar
Gauss, C. F. (1809), Theoria motus corporum coelestium in sectionibus conicis solem ambientium, Cambridge Library Collection, Cambridge University Press. 2011 reprint of the 1809 original.10.1017/CBO9780511841705CrossRefGoogle Scholar
Gauss, C. F. and Stewart, G. W. (1995), Theory of the Combination of Observations Least Subject to Errors Part One, Part Two, Supplement, SIAM.10.1137/1.9781611971248CrossRefGoogle Scholar
Gentleman, W. M. (1976), Row elimination for solving sparse linear systems and least squares problems, in Numerical Analysis (Watson, G. A., ed.), Vol. 506 of Lecture Notes in Mathematics, Springer, pp. 122–133.10.1007/BFb0080119CrossRefGoogle Scholar
George, A. (1973), Nested dissection of a regular finite element mesh, SIAM J. Numer. Anal. 10, 345–363.10.1137/0710032CrossRefGoogle Scholar
George, A. and Heath, M. T. (1980), Solution of sparse linear least squares problems using Givens rotations, Linear Algebra Appl. 34, 69–83.10.1016/0024-3795(80)90159-7CrossRefGoogle Scholar
George, A. and E. Ng (1983), On row and column orderings for sparse least squares problems, SIAM J. Numer. Anal. 20, 326–344.10.1137/0720022CrossRefGoogle Scholar
George, A., Liu, J. and E. Ng (1986), Row-ordering schemes for sparse Givens transformations, III: Analyses for a model problem, Linear Algebra Appl. 75, 225–240.10.1016/0024-3795(86)90191-6CrossRefGoogle Scholar
George, A., Liu, J. and E. Ng (1988), A data structure for sparse QR and LU factorizations, SIAM J. Sci. Comput. 9, 100–121.10.1137/0909008CrossRefGoogle Scholar
Gibbs, N. E., Poole, W. G. J. and Stockmeyer, P. K. (1976), An algorithm for reducing the bandwidth and profile of a sparse matrix, SIAM J. Numer. Anal. 13, 236–250.10.1137/0713023CrossRefGoogle Scholar
Gilbert, J. R., Li, X. S., Ng, E. G. and Peyton, B. W. (2001), Computing row and column counts for sparse QR and LU factorization, BIT Numer. 41, 693–710.10.1023/A:1021943902025CrossRefGoogle Scholar
Gill, P. E., Saunders, M. A. and Shinnerl, J. R. (1996), On the stability of Cholesky factorization for symmetric quasidefinite systems, SIAM J. Matrix Anal. Appl. 17, 35–46.10.1137/S0895479893252623CrossRefGoogle Scholar
Giraud, L. and Gratton, S. (2006), On the sensitivity of some spectral preconditioners, SIAM J. Matrix Anal. Appl. 27, 1089–1105.10.1137/040617546CrossRefGoogle Scholar
Giraud, L., Langou, J., Rozložnk, M. and van den Eshof, J. (2005), Rounding error analysis of the classical Gram–Schmidt orthogonalization process, Numer. Math. 101, 87–100.10.1007/s00211-005-0615-4CrossRefGoogle Scholar
Givens, W. (1953), A method of computing eigenvalues and eigenvectors suggested by classical results on symmetric matrices, in Simultaneous Linear Equations and the Determination of Eigenvalues , Vol. 29 of National Bureau of Standards Applied Mathematics Series, US Government Printing Office, pp. 117–122.Google Scholar
Gnanasekaran, A. and Darve, E. (2022), Hierarchical orthogonal factorization: Sparse least squares problems, J. Sci. Comput. 91, art. 50.10.1007/s10915-022-01824-9CrossRefGoogle Scholar
Golub, G. H. (1965), Numerical methods for solving linear least squares problems, Numer . Math. 7, 206–216.Google Scholar
Golub, G. H. and Meurant, G. (1997), Matrices, moments and quadrature, II: How to compute the norm of the error in iterative methods, BIT Numer. 37, 687–705.10.1007/BF02510247CrossRefGoogle Scholar
Golub, G. H. and Van Loan, C. F. (1996), Matrix Computations, fourth edition, Johns Hopkins University Press.Google Scholar
Golub, G. H. and Wilkinson, J. H. (1966), Note on the iterative refinement of least squares solution, Numer. Math. 9, 139–148.10.1007/BF02166032CrossRefGoogle Scholar
Goreinov, S. A., Tyrtyshnikov, E. E. and Zamarashkin, N. L. (1997), A theory of pseudoskeleton approximations, Linear Algebra Appl. 261, 1–21.10.1016/S0024-3795(96)00301-1CrossRefGoogle Scholar
Gould, N. I. M. and Scott, J. A. (2017), The state-of-the-art of preconditioners for sparse linear least-squares problems, ACM Trans. Math. Softw. 43, art. 36.10.1145/3014057CrossRefGoogle Scholar
Gratton, S., Gürol, S., Simon, E. and Toint, P. L. (2018), A note on preconditioning weighted linear least-squares, with consequences for weakly constrained variational data assimilation, Quart . J. R. Meteorol. Soc. 144, 934–940.10.1002/qj.3262CrossRefGoogle Scholar
Gratton, S., Lawless, A. S. and Nichols, N. K. (2007), Approximate Gauss–Newton methods for nonlinear least squares problems, SIAM J. Optim. 18, 106–132.10.1137/050624935CrossRefGoogle Scholar
Gratton, S., Sartenaer, A. and Tshimanga, J. (2011), On a class of limited memory preconditioners for large scale linear systems with multiple right-hand sides, SIAM J. Opt. 21, 912–935.10.1137/08074008CrossRefGoogle Scholar
Greif, C., He, S. and Liu, P. (2017), SYM-ILDL: Incomplete LDL T factorization of symmetric indefinite and skew-symmetric matrices, ACM Trans. Math. Softw. 44, art. 1.10.1145/3054948CrossRefGoogle Scholar
Hagemann, M. and Schenk, O. (2006), Weighted matchings for preconditioning symmetric indefinite linear systems, SIAM J. Sci. Comput. 28, 403–420.10.1137/040615614CrossRefGoogle Scholar
Hager, W. W. (1989), Updating the inverse of a matrix, SIAM Rev. 31, 221–239.10.1137/1031049CrossRefGoogle Scholar
Hansen, P. C. (1998), Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion, SIAM.10.1137/1.9780898719697CrossRefGoogle Scholar
Hansen, P. C. (2010), Discrete Inverse Problems : Insight and Algorithms, Vol. 7 of Fundamentals of Algorithms, SIAM.Google Scholar
Hansen, P. C., Hayami, K. and Morikuni, K. (2022), GMRES methods for tomographic reconstruction with an unmatched back projector, J. Comput. Appl. Math. 413, art. 114352.10.1016/j.cam.2022.114352CrossRefGoogle Scholar
Hansen, P. C., Pereyra, V. and Scherer, G. (2013), Least Squares Data Fitting with Applications, Johns Hopkins University Press.Google Scholar
Hanson, R. J. and Lawson, C. L. (1969), Extensions and applications of the Householder algorithm for solving linear least squares problems, Math. Comp. 23, 787–812.10.1090/S0025-5718-1969-0258258-9CrossRefGoogle Scholar
Hare, D. R., Johnson, C. R., Olesky, D. D. and van den Driessche, P. (1993), Sparsity analysis of the QR factorization, SIAM J. Matrix Anal. Appl. 14, 655–669.10.1137/0614046CrossRefGoogle Scholar
Hayami, K., Yin, J.-F. and Ito, T. (2010), GMRES methods for least squares problems, SIAM J. Matrix Anal. Appl. 31, 2400–2430.10.1137/070696313CrossRefGoogle Scholar
Heath, M. T. (1982), Some extensions of an algorithm for sparse linear least squares problems, SIAM J. Sci. Statist. Comput. 3, 223–237.10.1137/0903014CrossRefGoogle Scholar
Hénon, P., Ramet, P. and Roman, J. (2002), PaStiX: A high-performance parallel direct solver for sparse symmetric positive definite systems, Parallel Comput. 28, 301–321.10.1016/S0167-8191(01)00141-7CrossRefGoogle Scholar
Hestenes, M. R. and Stiefel, E. (1952), Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bureau Standards 49, 409–436.10.6028/jres.049.044CrossRefGoogle Scholar
Higham, N. J. (1990), Analysis of the Cholesky decomposition of a semi-definite matrix, in Reliable Numerical Computation (Cox, M. G. and Hammarling, S. J., eds), Oxford University Press, pp. 161–185.10.1093/oso/9780198535645.003.0010CrossRefGoogle Scholar
Higham, N. J. (2002), Accuracy and Stability of Numerical Algorithms, second edition, SIAM.10.1137/1.9780898718027CrossRefGoogle Scholar
Higham, N. J. and Mary, T. (2022), Mixed precision algorithms in numerical linear algebra, Acta Numer. 31, 347–414.10.1017/S0962492922000022CrossRefGoogle Scholar
Higham, N. J. and Pranesh, S. (2021), Exploiting lower precision arithmetic in solving symmetric positive definite linear systems and least squares problems, SIAM J. Sci. Comput. 43, A258–A277.10.1137/19M1298263CrossRefGoogle Scholar
Higham, N. J. and Stewart, G. W. (1987), Numerical linear algebra in statistical computing, in The State of the Art in Numerical Analysis (Iserles, I. and Powell, M. J. D., eds), Oxford University Press, pp. 41–57.Google Scholar
Hnětynková, I., Kubínová, M. and Plešinger, M. (2017), Noise representation in residuals of LSQR, LSMR, and CRAIG regularization, Linear Algebra Appl. 533, 357–379.10.1016/j.laa.2017.07.031CrossRefGoogle Scholar
Hogg, J. D. and Scott, J. A. (2013a), An efficient analyse phase for element problems, Numer . Linear Algebra Appl. 20, 397–412.10.1002/nla.1810CrossRefGoogle Scholar
Hogg, J. D. and Scott, J. A. (2013b), New parallel sparse direct solvers for multicore archiectures, Algorithms 6, 702–725.10.3390/a6040702CrossRefGoogle Scholar
Hogg, J. D. and Scott, J. A. (2013c), Pivoting strategies for tough sparse indefinite systems, ACM Trans. Math. Softw. 40, art. 4.10.1145/2513109.2513113CrossRefGoogle Scholar
Hogg, J. D., Reid, J. K. and Scott, J. A. (2010), Design of a multicore sparse Cholesky factorization using DAGs, SIAM J. Sci. Comput. 32, 3627–3649.10.1137/090757216CrossRefGoogle Scholar
Hogg, J. D., Scott, J. A. and Thorne, S. (2017), Numerically aware orderings for sparse symmetric indefinite linear systems, ACM Trans. Math. Softw. 44, art. 13.Google Scholar
Hong, Y. P. and Pan, C.-T. (1992), Rank-revealing QR factorizations and the singular value decomposition, Math. Comp. 58, 213–232.Google Scholar
Householder, A. S. (1958), Unitary triangularization of a nonsymmetric matrix, J. Assoc. Comput. Mach. 5, 339–342.10.1145/320941.320947CrossRefGoogle Scholar
Howell, G. W. and Baboulin, M. (2016), LU preconditioning for overdetermined sparse least squares problems, in Parallel Processing and Applied Mathematics, Part I, Vol. 9573 of Lecture Notes in Computer Science, Springer, pp. 128–137.10.1007/978-3-319-32149-3_13CrossRefGoogle Scholar
Hysom, D. and Pothen, A. (2002), Level-based incomplete LU factorization: Graph model and algorithms. Technical report UCRL-JC-150789, Lawrence Livermore National Laboratory, California, USA.Google Scholar
Ipsen, I. C. F. and Wentworth, T. (2014), The effect of coherence on sampling from matrices with orthonormal columns, and preconditioned least squares problems, SIAM J. Matrix Anal. Appl. 35, 1490–1520.10.1137/120870748CrossRefGoogle Scholar
Jacobi, C. G. J. (1845), Über eine neue Auflösungsart der bei der Methode der kleinsten Quadrate vorkommenden lineären Gleichungen, Astronom. Nachr. 523, 297–306.10.1002/asna.18450222002CrossRefGoogle Scholar
Jennings, A. and Ajiz, M. A. (1984), Incomplete methods for solving , SIAM J. Sci. Statist. Comput. 5, 978–987.10.1137/0905067CrossRefGoogle Scholar
Jin, S., Pei, S., Wang, Y. and Qi, Y. (2021), A parallel sparse triangular solve algorithm based on dependency elimination of the solution vector, Cluster Comput. 24, 1317–1330.10.1007/s10586-020-03188-xCrossRefGoogle Scholar
Jones, M. T. and Plassmann, P. E. (1995), An improved incomplete Cholesky factorization, ACM Trans. Math. Softw. 21, 5–17.10.1145/200979.200981CrossRefGoogle Scholar
Karypis, G. and Kumar, V. (1998), A fast and high quality multilevel scheme for partitioning irregular graphs, SIAM J. Sci. Comput. 20, 359–392.10.1137/S1064827595287997CrossRefGoogle Scholar
Kershaw, D. S. (1978), The incomplete Cholesky-conjugate gradient method for the iterative solution of systems of linear equations, J. Comput. Phys. 26, 43–65.10.1016/0021-9991(78)90098-0CrossRefGoogle Scholar
Kuřátko, J. (2019), Factorization of saddle-point matrices in dynamical systems optimization: Reusing pivots, Linear Algebra Appl. 566, 61–85.10.1016/j.laa.2018.12.026CrossRefGoogle Scholar
Laub, A. J. (2005), Matrix Analysis for Scientists & Engineers, SIAM.Google Scholar
Läuchli, P. (1961), Jordan-Elimination und Ausgleichung nach kleinsten Quadraten, Numer . Math. 3, 226–240.Google Scholar
Lawson, C. L. and Hanson, R. J. (1995), Solving Least Squares Problems , Vol. 15 of Classics in Applied Mathematics, SIAM. Revised reprint of the 1974 original.Google Scholar
Legendre, A. M. (1805), Nouvelles Méthodes pour la Détermination des Orbites des Comètes, Vol. 116 of Libraire pour les Mathématiques, la Marine, l’Architecture, Firmin Didot.Google Scholar
Li, N. and Saad, Y. (2005), Crout versions of ILU factorization with pivoting for sparse symmetric matrices, Electron . Trans. Numer. Anal. 20, 75–85.Google Scholar
Li, N. and Saad, Y. (2006), MIQR: A multilevel incomplete QR preconditioner for large sparse least-squares problems, SIAM J. Matrix Anal. Appl. 28, 524–550.10.1137/050633032CrossRefGoogle Scholar
Li, N., Saad, Y. and Chow, E. (2003), Crout versions of ILU for general sparse matrices, SIAM J. Sci. Comput. 25, 716–728.10.1137/S1064827502405094CrossRefGoogle Scholar
Liesen, J. and Strakoš, Z. (2013), Krylov Subspace Methods: Principles and Analysis, Numerical Mathematics and Scientific Computation, Oxford University Press.Google Scholar
Lipton, R. J., Rose, D. J. and Tarjan, R. E. (1979), Generalized nested dissection, SIAM J. Numer. Anal. 16, 346–358.10.1137/0716027CrossRefGoogle Scholar
Liu, J. W. H. (1986a), A compact row storage scheme for Cholesky factors using elimination trees, ACM Trans. Math. Softw. 12, 127–148.10.1145/6497.6499CrossRefGoogle Scholar
Liu, J. W. H. (1986b), On general row merging schemes for sparse Givens transformations, SIAM J. Sci. Comput. 7, 1190–1211.10.1137/0907081CrossRefGoogle Scholar
Liu, J. W. H. (1989), The minimum degree ordering with constraints, SIAM J. Sci. Comput. 10, 1136–1145.10.1137/0910069CrossRefGoogle Scholar
Liu, J. W. H. (1990), The role of elimination trees in sparse factorizations, SIAM J. Matrix Anal. Appl. 11, 134–172.10.1137/0611010CrossRefGoogle Scholar
Liu, J. W. H. (1992), The multifrontal method for sparse matrix solution: Theory and practice, SIAM Rev. 34, 82–109.10.1137/1034004CrossRefGoogle Scholar
Lukšan, L. and Vlček, J. (1998), Indefinitely preconditioned inexact Newton method for large sparse equality constrained non-linear programming problems, Numer . Linear Algebra Appl. 5, 219–247.10.1002/(SICI)1099-1506(199805/06)5:3<219::AID-NLA134>3.0.CO;2-73.0.CO;2-7>CrossRefGoogle Scholar
Magnus, J. R. (2022), Gauss on least-squares and maximum-likelihood estimation, Arch . Hist. Exact Sci. 76, 425–430.10.1007/s00407-022-00291-wCrossRefGoogle Scholar
Mandel, J. (1993), Balancing domain decomposition, Commun . Numer. Methods Engrg 9, 233–241.10.1002/cnm.1640090307CrossRefGoogle Scholar
Marín, J., Mas, J., Guerrero, D. and Hayami, K. (2017), Updating preconditioners for modified least squares problems, Numer . Algorithms 75, 491–508.10.1007/s11075-017-0315-zCrossRefGoogle Scholar
Markowitz, H. M. (1957), The elimination form of the inverse and its application to linear programming, Manag . Sci. 3, 255–269.Google Scholar
Martinsson, P.-G. and Tropp, J. A. (2020), Randomized numerical linear algebra: Foundations and algorithms, Acta Numer. 29, 403–572.10.1017/S0962492920000021CrossRefGoogle Scholar
Martinsson, P.-G., Quintana-Ortí, G., Heavner, N. and van de Geijn, R. (2017), Householder QR factorization with randomization for column pivoting (HQRRP), SIAM J. Sci. Comput. 39, C96–C115.10.1137/16M1081270CrossRefGoogle Scholar
Matstoms, P. (1994), Sparse QR Factorization with Applications to Linear Least Squares Problems, Vol. 337 of Linköping Studies in Science and Technology, Linköping University, Department of Mathematics.Google Scholar
Meier, M. and Nakatsukasa, Y. (2022), Randomized algorithms for Tikhonov regularization in linear least squares. Available at arXiv:2203.07329.Google Scholar
Meier, M., Nakatsukasa, Y., Townsend, A. and Webb, M. (2024), Are sketch-and-precondition least squares solvers numerically stable?, SIAM J. Matrix Anal. Appl. 45, 905–929.10.1137/23M1551973CrossRefGoogle Scholar
Melnichenko, M., Balabanov, O., Murray, R., Demmel, J., Mahoney, M. W. and Luszczek, P. (2024), CholeskyQR with randomization and pivoting for tall matrices (CQRRPT). Available at arXiv:2311.08316.Google Scholar
Meng, X., Saunders, M. A. and Mahoney, M. W. (2014), LSRN: A parallel iterative solver for strongly over-or underdetermined systems, SIAM J. Sci. Comput. 36, C95–C118.10.1137/120866580CrossRefGoogle ScholarPubMed
Murray, R., Demmel, J., Mahoney, M. W., Erichson, N. B., Melnichenko, M., Malik, O. A., Grigori, L., Luszczek, P., Dereziński, M., Lopes, M. E., Liang, T., Luo, H. and Dongarra, J. (2023), Randomized numerical linear algebra: A perspective on the field with an eye to software. Available at arXiv:2302.11474.Google Scholar
Napov, A. (2023), An incomplete Cholesky preconditioner based on orthogonal approximations, SIAM J. Sci. Comput. 45, A729–A752.10.1137/21M1468334CrossRefGoogle Scholar
E. Ng (1991), A scheme for handling rank-deficiency in the solution of sparse linear least squares problems, SIAM J. Sci. Comput. 12, 1173–1183.10.1137/0912062CrossRefGoogle Scholar
Ng, E. G. and Peyton, B. W. (1992), A tight and explicit representation of Q in sparse QR factorization. Report ORNL/TM–12059, Oak Ridge National Laboratory, TN.Google Scholar
Ng, E. G. and Peyton, B. W. (1993a), Block sparse Cholesky algorithms on advanced uniprocessor computers, SIAM J. Sci. Comput. 14, 1034–1056.10.1137/0914063CrossRefGoogle Scholar
Ng, E. G. and Peyton, B. W. (1993b), A supernodal Cholesky factorization algorithm for shared-memory multiprocessors, SIAM J. Sci. Comput. 14, 761–769.10.1137/0914048CrossRefGoogle Scholar
Oliveira, A. R. L. and Sorensen, D. C. (2005), A new class of preconditioners for large-scale linear systems from interior point methods for linear programming, Linear Algebra Appl. 394, 1–24.10.1016/j.laa.2004.08.019CrossRefGoogle Scholar
Orban, D. (2015), Limited-memory LDL T factorization of symmetric quasi-definite matrices with application to constrained optimization, Numer . Algorithms 70, 9–41.10.1007/s11075-014-9933-xCrossRefGoogle Scholar
Orban, D. and Arioli, M. (2017), Iterative Solution of Symmetric Quasi-Definite Linear Systems, SIAM.10.1137/1.9781611974737CrossRefGoogle Scholar
Osinsky, A. I. and Zamarashkin, N. L. (2018), Pseudo-skeleton approximations with better accuracy estimates, Linear Algebra Appl. 537, 221–249.10.1016/j.laa.2017.09.032CrossRefGoogle Scholar
Ozaslan, I. K., Pilanci, M. and Arikan, O. (2023), M-IHS: An accelerated randomized preconditioning method avoiding costly matrix decompositions, Linear Algebra Appl. 678, 57–91.10.1016/j.laa.2023.08.014CrossRefGoogle Scholar
Paige, C. C. and Saunders, M. A. (1975), Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12, 617–629.10.1137/0712047CrossRefGoogle Scholar
Paige, C. C. and Saunders, M. A. (1982), LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Softw. 8, 43–71.10.1145/355984.355989CrossRefGoogle Scholar
Papadopoulos, A. T., Duff, I. S. and Wathen, A. J. (2005), A class of incomplete orthogonal factorization methods, II: Implementation and results, BIT Numer. 45, 159–179.10.1007/s10543-005-2642-zCrossRefGoogle Scholar
Parter, S. (1961), The use of linear graphs in Gaussian elimination, SIAM Rev. 3, 119–130, 364–369.10.1137/1003021CrossRefGoogle Scholar
Pearson, J. W. and Pestana, J. (2020), Preconditioners for Krylov subspace methods: An overview, GAMM-Mitteilungen 43, art. e202000015.10.1002/gamm.202000015CrossRefGoogle Scholar
Peters, G. and Wilkinson, J. H. (1970), The least squares problem and pseudo-inverse, Comput. J. 131, 309–316.10.1093/comjnl/13.3.309CrossRefGoogle Scholar
Pierce, D. J. and Lewis, J. G. (1997), Sparse multifrontal rank revealing QR factorization, SIAM J. Matrix Anal. Appl. 18, 159–180.10.1137/S0895479893244353CrossRefGoogle Scholar
Pothen, A. (1993), Predicting the structure of sparse orthogonal factors, Linear Algebra Appl. 194, 183–203.10.1016/0024-3795(93)90121-4CrossRefGoogle Scholar
Pothen, A. and Fan, C. J. (1990), Computing the block triangular form of a sparse matrix, ACM Trans. Math. Softw. 16, 303–324.10.1145/98267.98287CrossRefGoogle Scholar
Rees, T. and Scott, J. A. (2018), A comparative study of null-space factorizations for sparse symmetric saddle point systems, Numer . Linear Algebra Appl. 25, art. e2103.10.1002/nla.2103CrossRefGoogle Scholar
Reichel, L., Sadok, H. and Zhang, W.-H. (2020), Simple stopping criteria for the LSQR method applied to discrete ill-posed problems, Numer . Algorithms 84, 1381–1395.10.1007/s11075-019-00852-1CrossRefGoogle Scholar
Reid, J. K. and Scott, J. A. (1999), Ordering symmetric sparse matrices for small profile and wavefront, Int. J. Numer. Method Engrg 45, 1737–1755.10.1002/(SICI)1097-0207(19990830)45:12<1737::AID-NME652>3.0.CO;2-T3.0.CO;2-T>CrossRefGoogle Scholar
Reid, J. K. and Scott, J. A. (2009), An out-of-core sparse Cholesky solver, ACM Trans. Math. Softw. 36, art. 9.10.1145/1499096.1499098CrossRefGoogle Scholar
Reißig, G. (2007), Local fill reduction techniques for sparse symmetric linear systems, Electr. Engrg 89, 639–652.10.1007/s00202-006-0042-2CrossRefGoogle Scholar
Rose, D. J. (1972), A graph-theoretic study of the numerical solution of sparse positive definite systems of linear equations, in Graph Theory and Computing (Read, R., ed.), Academic Press, pp. 183–217.10.1016/B978-1-4832-3187-7.50018-0CrossRefGoogle Scholar
Rose, D. J., Tarjan, R. E. and Lueker, G. S. (1976), Algorithmic aspects of vertex elimination on graphs, SIAM J. Comput. 5, 266–283.10.1137/0205021CrossRefGoogle Scholar
Rothberg, E. and Eisenstat, S. (1998), Node selection strategies for bottom-up sparse matrix ordering, SIAM J. Matrix Anal. Appl. 19, 682–695.10.1137/S0895479896302692CrossRefGoogle Scholar
Rozložník, M., Smoktunowicz, A. and Kopal, J. (2014), A note on iterative refinement for seminormal equations, Appl. Numer. Math. 75, 167–174.10.1016/j.apnum.2013.08.005CrossRefGoogle Scholar
Saad, Y. (2003a), Finding exact and approximate block structures for ILU preconditioning, SIAM J. Sci. Comput. 24, 1107–1123.10.1137/S1064827501393393CrossRefGoogle Scholar
Saad, Y. (2003b), Iterative Methods for Sparse Linear Systems, second edition, SIAM.10.1137/1.9780898718003CrossRefGoogle Scholar
Saad, Y. and Schultz, M. H. (1986), GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Comput. 7, 856–869.10.1137/0907058CrossRefGoogle Scholar
Saad, Y. and van der Vorst, H. A. (2000), Iterative solution of linear systems in the 20th century, J. Comput. Appl. Math. 123, 1–33.10.1016/S0377-0427(00)00412-XCrossRefGoogle Scholar
Saad, Y., Yeung, M., Erhel, J. and Guyomarch, F. (2000), A deflated version of the conjugate gradient algorithm, SIAM J. Sci. Comput. 21, 1909–1926.10.1137/S1064829598339761CrossRefGoogle Scholar
Saunders, M. A. (1979), Sparse least squares problems by conjugate gradients: A comparison of preconditioning methods, in Proceedings of Computer Science and Statistics: Twelfth Annual Conference on the Interface, University of Waterloo, Canada.Google Scholar
Saunders, M. A. (1995), Solution of sparse rectangular systems using LSQR and Craig, BIT Numer. 35, 588–604.10.1007/BF01739829CrossRefGoogle Scholar
Saunders, M. A. (1996), Cholesky-based methods for sparse least squares: The benefits of regularization, in Linear and Nonlinear Conjugate Gradient-Related Methods (Adams, L. and Nazareth, J. L., eds), SIAM, pp. 92–100.Google Scholar
Schenk, O. and Gärtner, K. (2006), On fast factorization pivoting methods for symmetric indefinite systems, Electron . Trans. Numer. Anal. 23, 158–179.Google Scholar
Schreiber, R. and Van Loan, C. (1989), A storage-efficient WY representation for products of Householder transformations, SIAM J. Sci. Comput. 10, 53–57.10.1137/0910005CrossRefGoogle Scholar
Scott, J. A. (2023), HSL@60: A brief history of the HSL mathematical software library. Technical report STFC-TR-2023-002, Science and Technology Facilities Council, Didcot.Google Scholar
Scott, J. A. and Tůma, M. (2011), The importance of structure in incomplete factorization preconditioners, BIT Numer. 51, 385–404.10.1007/s10543-010-0299-8CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2014a), HSL_MI28: An efficient and robust limited-memory incomplete Cholesky factorization code, ACM Trans. Math. Softw. 40, art. 24.10.1145/2617555CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2014b), On positive semidefinite modification schemes for incomplete Cholesky factorization, SIAM J. Sci. Comput. 36, A609–A633.10.1137/130917582CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2014c), On signed incomplete Cholesky factorization preconditioners for saddle-point systems, SIAM J. Sci. Comput. 36, A2984–A3010.10.1137/140956671CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2016), Preconditioning of linear least squares by robust incomplete factorization for implicitly held normal equations, SIAM J. Sci. Comput. 38, C603–C623.10.1137/16M105890XCrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2017a), Improving the stability and robustness of incomplete symmetric indefinite factorization preconditioners, Numer . Linear Algebra Appl. 24, art. e2099.10.1002/nla.2099CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2017b), Solving mixed sparse-dense linear least-squares problems by preconditioned iterative methods, SIAM J. Sci. Comput. 39, A2422–A2437.10.1137/16M1108339CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2019), Sparse stretching for solving sparse-dense linear least-squares problems, SIAM J. Sci. Comput. 41, A1604–A1625.10.1137/18M1181353CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2021), Strengths and limitations of stretching for least-squares problems with some dense rows, ACM Trans. Math. Softw. 47, art. 1.10.1145/3412559CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2022a), A computational study of using black-box QR solvers for large-scale sparse-dense linear least squares problems, ACM Trans. Math. Softw. 48, art. 5.10.1145/3494527CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2022b), A null-space approach for large-scale symmetric saddle point systems with a small and non zero (2, 2) block, Numer . Algorithms 90, 1639–1667.10.1007/s11075-021-01245-zCrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2022c), Solving large linear least squares problems with linear equality constraints, BIT Numer. 62, 1765–1787.10.1007/s10543-022-00930-2CrossRefGoogle Scholar
Scott, J. A. and Tůma, M. (2023), Algorithms for Sparse Linear Systems, Nečas Center Series, Birkhäuser/Springer.10.1007/978-3-031-25820-6CrossRefGoogle Scholar
Scott, J. and Tůma, M. (2024), Avoiding breakdown in incomplete factorizations in low precision arithmetic, ACM Trans. Math. Software 50, art. 9.10.1145/3651155CrossRefGoogle Scholar
Scott, J. and Tůma, M. (2025), Developing robust incomplete Cholesky factorization in half precision arithmetic, Numer. Algorithms. Available at https://doi.org/10.1007/s11075-025-02015-x.CrossRefGoogle Scholar
di Perrotolo, A. Scotto (2022), Randomized numerical linear algebra methods with application to data assimilation. PhD thesis, Institut Superieur de l’Aeronautique et de l’Espace, Toulouse.Google Scholar
Sloan, S. W. (1986), An algorithm for profile and wavefront reduction of sparse matrices, Int. J. Numer. Method Engrg 23, 239–251.10.1002/nme.1620230208CrossRefGoogle Scholar
Sorensen, D. C. (1977), Updating the symmetric indefinite factorization with applications in a modified Newton’s method. PhD thesis, University of California, San Diego.Google Scholar
Speelpenning, B. (1978), Generalized element method. Technical report, Department of Computer Science, Illinois University, Urbana, IL.Google Scholar
Spielman, D. A. and Teng, S.-H. (2014), Nearly linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems, SIAM J. Matrix Anal. Appl. 35, 835–885.10.1137/090771430CrossRefGoogle Scholar
Tang, J. M., Nabben, R., Vuik, C. and Erlangga, Y. A. (2009), Comparison of two-level preconditioners derived from deflation, domain decomposition and multigrid methods, J. Sci. Comput. 39, 340–370.10.1007/s10915-009-9272-6CrossRefGoogle Scholar
Tarjan, R. E. (1975), Efficiency of a good but not linear set union algorithm, J. Assoc. Comput. Mach. 22, 215–225.10.1145/321879.321884CrossRefGoogle Scholar
Terao, T., Ozaki, K. and Ogita, T. (2020), LU-Cholesky QR algorithms for thin QR decomposition, Parallel Comput. 92, art. 102571.10.1016/j.parco.2019.102571CrossRefGoogle Scholar
Tinney, W. F. and Walker, J. W. (1967), Direct solutions of sparse network equations by optimally ordered triangular factorization, Proc . IEEE 55, 1801–1809.10.1109/PROC.1967.6011CrossRefGoogle Scholar
Tůma, M. (2002), A note on the LDL T decomposition of matrices from saddle-point problems, SIAM J. Matrix Anal. Appl. 23, 903–925.10.1137/S0895479897321088CrossRefGoogle Scholar
Van der Sluis, A. (1969), Condition numbers and equilibration of matrices, Numer . Math. 14, 14–23.Google Scholar
van der Vorst, H. A. (2003), Iterative Krylov Methods for Large Linear Systems, Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press.10.1017/CBO9780511615115CrossRefGoogle Scholar
Van Loan, C. (1985), On the method of weighting for equality-constrained least-squares problems, SIAM J. Numer. Anal. 22, 851–864.10.1137/0722051CrossRefGoogle Scholar
Vanderbei, R. J. (1995), Symmetric quasidefinite matrices, SIAM J. Optim. 5, 100–113.10.1137/0805005CrossRefGoogle Scholar
Varga, R. S. (1960), Factorization and normalized iterative methods, in Boundary Problems in Differential Equations (Langer, R. E., ed.), University of Wisconsin Press, pp. 121–142.Google Scholar
Wang, X., Gallivan, K. A. and Bramley, R. (1997), CIMGS: An incomplete orthogonal factorization preconditioner, SIAM J. Sci. Comput. 18, 516–536.10.1137/S1064827594268270CrossRefGoogle Scholar
Wathen, A. J. (2015), Preconditioning, Acta Numer. 24, 329–376.10.1017/S0962492915000021CrossRefGoogle Scholar
Wathen, A. J. (2022), Some comments on preconditioning for normal equations and least squares, SIAM Rev. 64, 640–649.10.1137/20M1387948CrossRefGoogle Scholar
Watts, J. W. III (1981), A conjugate gradient truncated direct method for the iterative solution of the reservoir simulation pressure equation, Soc . Petroleum Engrg J. 21, 345–353.Google Scholar
Wedin, P. (1973), Perturbation theory for pseudo-inverses, BIT Numer. 13, 217–232.10.1007/BF01933494CrossRefGoogle Scholar
Wilkinson, J. H. (1968), A priori error analysis of algebraic processes, in Proc. Internat. Congr. Math. (Moscow, 1966), Izdat. ‘Mir’, pp. 629–640.Google Scholar
Woodbury, M. A. (1949), The stability of out–input matrices. Chicago, IL.Google Scholar
Woodbury, M. A. (1950), Inverting modified matrices. Memorandum Report 42, Statistical Research Group, Princeton University, NJ.Google Scholar
Yamamoto, Y., Nakatsukasa, Y., Yanagisawa, Y. and Fukaya, T. (2015), Roundoff error analysis of the CholeskyQR2 algorithm, Electron . Trans. Numer. Anal. 44, 306–326.Google Scholar
Yeralan, S. N., Davis, T. A., Sid-Lakhdar, W. M. and Ranka, S. (2017), Algorithm 980: Sparse QR factorization on the GPU, ACM Trans. Math. Softw. 44, art. 17.Google Scholar
Zhao, T. (2016), A spectral analysis of subspace enhanced preconditioners, J. Sci. Comput. 66, 435–457.10.1007/s10915-015-0029-0CrossRefGoogle Scholar
Zilli, G. and Bergamaschi, L. (2022), Block preconditioners for linear systems in interior point methods for convex constrained optimization, Ann . Univ. Ferrara Sez. VII Sci. Mat. 68, 337–368.10.1007/s11565-022-00422-9CrossRefGoogle Scholar