Skip to main content
SpringerLink
Log in

Low complexity matrix projections preserving actions on vectors

  • Published:
Calcolo Aims and scope Submit manuscript

Abstract

In this paper we prove that, given a \(n \times n\) symmetric matrix A, a matrix V with r orthonormal columns and an integer \(m \ge 1\), \(mr \le n\), it is possible to devise a matrix algebra \(\mathcal {L}\) such that, denoting by \(\mathcal {L}_A\) the matrix closest to A from \(\mathcal {L}\) in the Frobenius norm, one has \(\mathcal {L}^j_AV=A^jV\) for \(j=0,\dots ,m-1\). The algebra \(\mathcal {L}\) is the space of all matrices that are diagonalized by a given orthogonal matrix L. We show, moreover, that L can be constructed as the product of mr Householder matrices, so that \(\mathcal {L}\), for \(mr \ll n\), is a low complexity matrix algebra. The new theoretical results here introduced allow to investigate new possible preconditioners \({\mathcal {L}}_A\) for the Conjugate Gradient method and new quasi-Newton algorithms suitable to solve large scale optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Bell, N., Garland, M.: Cusp: Generic Parallel Algorithms for Sparse Matrix and Graph Computations. Version 0.5.1. http://cusplibrary.github.io/ (2015)

  2. Benzi, M.: Preconditioning techniques for large linear systems: a survey. J. Comput. Phys. 182(2), 418–477 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bergamaschi, L., Martínez, Á.: Two-stage spectral preconditioners for iterative eigensolvers. Numer. Linear Algebra Appl. 24(3), e2084 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bertaccini, D., Durastante, F.: Iterative Methods and Preconditioning for Large and Sparse Linear Systems with Applications. Chapman and Hall/CRC, Boca Raton (2018)

    Book  MATH  Google Scholar 

  5. Bortoletti, A., Di Fiore, C., Fanelli, S., Zellini, P.: A new class of quasi-Newtonian methods for optimal learning in MLP-networks. IEEE Trans. Neural Netw. Learn. Syst. 14(2), 263–273 (2003)

    Article  Google Scholar 

  6. Bridson, R., Tang, W.P.: Multiresolution approximate inverse preconditioners. SIAM J. Sci. Comput. 23(2), 463–479 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  7. Byrd, R.H., Nocedal, J., Yuan, Y.X.: Global convergence of a cass of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24(5), 1171–1190 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  8. Capizzano, S.S.: Superlinear PCG methods for symmetric Toeplitz systems. Math. Comput. 68, 793–803 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chen, J., Saad, Y.: Lanczos vectors versus singular vectors for effective dimension reduction. IEEE Trans. Knowl. Data Eng. 21(8), 1091–1103 (2009)

    Article  Google Scholar 

  10. Cipolla, S., Di Fiore, C., Zellini, P.: Updating Broyden Class-type descent directions by Householder adaptive Transforms (2018)

  11. Cipolla, S., Di Fiore, C., Tudisco, F., Zellini, P.: Adaptive matrix algebras in unconstrained minimization. Linear Algebra Appl. 471, 544–568 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Cipolla, S., Di Fiore, C., Tudisco, F.: Euler–Richardson method preconditioned by weakly stochastic matrix algebras: a potential contribution to Pagerank computation. Electron. J. Linear Algebra 32, 254–272 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Cipolla, S., Di Fiore, C., Durastante, F., Zellini, P.: Regularizing properties of a class of matrices including the optimal and the superoptimal preconditioners. Numer. Linear Algebra Appl. 26, e2225 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  14. Davis, T.A., Hu, Y.: The university of Florida sparse matrix collection. ACM Trans. Math. Softw. (TOMS) 38(1), 1–25 (2011)

    MathSciNet  MATH  Google Scholar 

  15. Di Benedetto, F., Capizzano, S.S.: A unifying approach to abstract matrix algebra preconditioning. Numer. Math. 82, 57–90 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  16. Di Benedetto, F., Estatico, C., Serra Capizzano, S.: Superoptimal preconditioned conjugate gradient iteration for image deblurring. SIAM J. Sci. Comput. 26(3), 1012–1035 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  17. Di Fiore, C.: Structured matrices in unconstrained minimization methods. In: Olshevsky, V., (ed.) Fast Algorithms for Structured Matrices: Theory and Applications, Contemporary Mathematics, vol. 323. AMS SIAM (2003)

  18. Di Fiore, C., Zellini, P.: Matrix algebras in optimal preconditioning. Linear Algebra Appl. 335, 1–54 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. Di Fiore, C., Fanelli, S., Lepore, F., Zellini, P.: Matrix algebras in quasi-Newton methods for unconstrained minimization. Numer. Math. 94, 479–500 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  20. Di Fiore, C., Lepore, F., Zellini, P.: Hartley-type algebras in displacement and optimization strategies. Linear Algebra Appl. 366, 215–232 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Di Fiore, C., Fanelli, S., Zellini, P.: Low-complexity minimization algorithms. Numer. Linear Algebra Appl. 12(8), 755–768 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  22. Estatico, C.: A class of filtering superoptimal preconditioners for highly ill-conditioned linear systems. BIT 42(4), 753–778 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  23. Fezzani, R., Grigori, L., Nataf, F., Wang, K.: Block filtering decomposition. Numer. Linear Algebra Appl. 21(6), 703–721 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  24. Freitag, M.A., Spence, A.: A tuned preconditioner for inexact inverse iteration applied to Hermitian eigenvalue problems. IMA J. Numer. Anal. 28(3), 522–551 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  25. Golub, G.H., Van Loan, C.F.: Matrix Computations, vol. 3. JHU Press, Baltimore (2012)

    MATH  Google Scholar 

  26. Goreinov, S.A., Tyrtyshnikov, E.E., Zamarashkin, N.L.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261, 1–21 (1997a)

    Article  MathSciNet  MATH  Google Scholar 

  27. Goreinov, S.A., Zamarashkin, N.L., Tyrtyshnikov, E.E.: Pseudo-skeleton approximations by matrices of maximal volume. Math. Notes 62, 515–519 (1997b)

    Article  MathSciNet  MATH  Google Scholar 

  28. Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern Mathematical Models, Methods and Algorithms for Real World Systems, pp. 420–447. Anamaya Publishers, New Delhi (2007)

    Google Scholar 

  29. Hanke, M., Nagy, J., Plemmons, R.: Preconditioned iterative regularization for lll-posed problems. Retrieved from the University of Minnesota Digital Conservancy (1992). http://hdl.handle.net/11299/2169

  30. Horn, R.A., Johnson, C.R.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)

    MATH  Google Scholar 

  31. Jin, X.Q., Wei, Y.M.: A survey and some extensions of T. Chan’s preconditioner. Linear Algebra Appl. 428(2), 403–412 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. Kailath, T., Sayed, A.H.: Fast Reliable Algorithms for Matrices with Structure. SIAM, Philadelphia (1999)

    Book  MATH  Google Scholar 

  33. Lopez, L., Simoncini, V.: Preserving geometric properties of the exponential matrix by block Krylov subspace methods. BIT 46(4), 813–830 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Martínez, Á.: Tuned preconditioners for the eigensolution of large spd matrices arising in engineering problems. Numer. Linear Algebra Appl. 23(3), 427–443 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Ming, G., Li, X.S., Vassilevski, P.S.: Direction-preserving and Schur-monotonic semiseparable approximations of symmetric positive definite matrices. SIAM J. Matrix Anal. Appl. 31, 2650–2664 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  36. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, Berlin (2006)

    MATH  Google Scholar 

  37. Osinsky, A., Zamarashkin, N.: Pseudo-skeleton approximations with better accuracy estimates. Linear Algebra Appl. 537, 221–249 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  38. Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. Nonlinear Program. 9, 53–72 (1976)

    MathSciNet  MATH  Google Scholar 

  39. Saad, Y.: Analysis of some Krylov subspace approximations to the matrix exponential operator. SIAM J. Numer. Anal. 29(1), 209–228 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  40. Saad, Y.: Iterative Methods for Sparse Linear Systems. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  41. Saad, Y.: Numerical Methods for Large Eigenvalue Problems. SIAM, Philadelphia (2011)

    Book  MATH  Google Scholar 

  42. Sadkane, M.: Block-Arnoldi and Davidson methods for unsymmetric large eigenvalue problems. Numer. Math. 64(1), 195–211 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  43. Tyrtyshnikov, E.E.: Optimal and superoptimal circulant preconditioners. SIAM J. Matrix Anal. Appl. 13(2), 459–473 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  44. Tyrtyshnikov, E.E.: A unifying approach to some old and new theorems on distribution and clustering. Linear Algebra Appl. 232, 1–43 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  45. Wagner, C.: Tangential frequency filtering decompositions for unsymmetric matrices. Numer. Math. 78(1), 143–163 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  46. Wagner, C., Wittum, G.: Adaptive filtering. Numer. Math. 78(2), 305–328 (1997)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

S. Cipolla acknowledges Dr. F. Durastante for pointing out Lemma 3.1 in [39] and for fruitful discussions on its generalization (Lemma 3). Moreover, he would like to thank Professor M.A. Botchev because the original idea of Theorem 1 was born during his lectures in the Rome-Moscow school of Matrix Methods and Applied Linear Algebra (2016 Edition). We acknowledge the anonymous referee for pointing relevant typos and mistakes and for his/her remarks which improved significantly the quality of the manuscript and leaded, in particular, to the statement of the result in Corollary 1. S.C. and C.D.F. are members of the INdAM Research group GNCS, which partially supported this work. The work of C.D.F. was partially supported by MIUR Excellence Department Project awarded to the Department of Mathematics, University of Rome Tor Vergata, CUP E83C18000100006.

Author information

Authors and Affiliations

  1. University of Padua, Via Trieste 63, 35121, Padua, Italy

    Stefano Cipolla

  2. University of Rome “Tor Vergata”, Via Della Ricerca Scientifica 1, 00133, Rome, Italy

    Carmine Di Fiore & Paolo Zellini

Authors
  1. Stefano Cipolla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carmine Di Fiore
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paolo Zellini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefano Cipolla.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cipolla, S., Di Fiore, C. & Zellini, P. Low complexity matrix projections preserving actions on vectors. Calcolo 56, 8 (2019). https://doi.org/10.1007/s10092-019-0305-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10092-019-0305-8

Keywords

Mathematics Subject Classification

Search

Navigation