Skip to main content
SpringerLink
Log in

The genesis and early developments of Aitken’s process, Shanks’ transformation, the ε–algorithm, and related fixed point methods

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, we trace back the genesis of Aitken’s Δ2 process and Shanks’ sequence transformation. These methods, which are extrapolation methods, are used for accelerating the convergence of sequences of scalars, vectors, matrices, and tensors. They had, and still have, many important applications in numerical analysis and in applied mathematics. They are related to continued fractions and Padé approximants. We go back to the roots of these methods and analyze the original contributions. New and detailed explanations on the building and properties of Shanks’ transformation and its kernel are provided. We then review their historical algebraic and algorithmic developments. We also analyze how they were involved in the solution of systems of linear and nonlinear equations, in particular in the methods of Steffensen, Pulay, and Anderson. Testimonies by various actors of the domain are given. The paper can also serve as an introduction to this domain of numerical analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abeles, F.F.: Chiò’s and Dodgson’s determinantal identities. Linear Algebra Appl. 454, 130–137 (2014)

    MathSciNet  MATH  Google Scholar 

  2. Aitken, A.C.: On Bernoulli’s numerical solution of algebraic equations. Proc. R. Soc. Edinb. 46, 289–305 (1925)

    MATH  Google Scholar 

  3. Aitken, A.C.: Further numerical studies in algebraic equations and matrices. Proc. R. Soc. Edinb. 51, 80–90 (1930)

    MATH  Google Scholar 

  4. Aitken, A.C.: Studies in practical mathematics. II. The evaluation of latent roots and latent vectors of matrix. Proc. R. Soc. Edinb. 57, 269–304 (1936)

    MATH  Google Scholar 

  5. Aitken, A.C.: Determinants and Matrices. Oliver and Boyd, Edinburgh (1951)

    MATH  Google Scholar 

  6. Aitken, A.C.: Gallipoli to the Somme: Recollections of a New Zealand Infantryman. Oxford University Press, Oxford (1963)

  7. Anderson, D.G.: Iterative procedures for nonlinear integral equations. J. Assoc. Comput. Mach. 12, 547–560 (1965)

    MathSciNet  MATH  Google Scholar 

  8. Anderson, D.G.M.: Comments on “Anderson Acceleration, Mixing and Extrapolation”, Numer. Algorithms, this issue

  9. Andrews, G.E., Goulden, I.P., Jackson, D.M.: Shanks convergence acceleration transform, Padé approximants and partitions. J. Combin. Theory Ser. A 43, 70–84 (1986)

    MathSciNet  MATH  Google Scholar 

  10. Arnoldi, W.E.: The principle of minimized iteration in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9, 17–29 (1951)

    MathSciNet  MATH  Google Scholar 

  11. Baron, G., Wajc, S.: Convergence acceleration of non–scalar sequences with non–linear transformations. Math. Comput. Simul. 23, 133–141 (1981)

    MATH  Google Scholar 

  12. Bauer, F.: Connections Between the q −d Algorithm of Rutishauser and the ε–Algorithm of Wynn, Deutsche Forschungsgemeinschaft Technical Report Ba/106 (1957)

  13. Bauer, F.L.: The quotient–difference and the epsilon algorithms. In: Langer, R.E. (ed.) On Numerical Approximation, pp. 361–370. The University of Madison Press, Madison (1959)

  14. Bauer, F.L.: The g–algorithm. J. Soc. Indust. Appl. Math. 8, 1–17 (1960)

    MathSciNet  MATH  Google Scholar 

  15. Bauer, F.L.: Nonlinear sequence transformations. In: Garabedian, H.L. (ed.) Approximation of Functions, pp. 134–151. Elsevier, Amsterdam (1965)

  16. Beckermann, B., Neuber, A., Mühlbach, G.: Shanks’ transformation revisited. Linear Algebra Appl. 173, 191–219 (1992)

    MathSciNet  MATH  Google Scholar 

  17. Berlinet, A.F.: Geometric approach to the parallel sum of vectors and application to the vector ε–algorithms. Numer. Algorithms 65, 783–807 (2014)

    MathSciNet  MATH  Google Scholar 

  18. Beuneu, J.: Résolution des Systèmes d’Équations Linéaires par la Méthode des Compensations, Technical Report ANO 69, Université de Lille I (1976)

  19. Beuneu, J.: Méthodes de Projection à Convergence Finie. Remarques sur leur Forme Incomplète, Tech. Report ANO 80, Université de Lille I (1982)

  20. Beuneu, J.: Méthodes de Projection–Minimisation pour les Systèmes Linéaires à Matrice Non Inversible, Tech. Report ANO 97, Université de Lille I (1983)

  21. Beuneu, J.: Méthodes de projection–minimisation pour les problèmes linéaires, RAIRO. Anal. Numér. 17, 221–248 (1983)

    MathSciNet  MATH  Google Scholar 

  22. Brezinski, C.: Application de l’ε–algorithme à la résolution des systèmes non linéaires. C. R. Acad. Sci. Paris 271 A, 1174–1177 (1970)

    MATH  Google Scholar 

  23. Brezinski, C.: Sur un algorithme de résolution des systèmes non linéaires. C. R. Acad. Sci. Paris 272 A, 145–148 (1971)

    MATH  Google Scholar 

  24. Brezinski, C.: Méthodes d’Accélération de la Convergence en Analyse Numérique, Thèse de Doctorat d’État ès Sciences Mathématiques, Université Scientifique et Médicale de Grenoble (1971)

  25. Brezinski, C.: Some results in the theory of the vector ε–algorithm. Linear Alg. Appl. 8, 77–86 (1974)

    MathSciNet  MATH  Google Scholar 

  26. Brezinski, C.: Généralisation de la transformation de Shanks, de la table de Padé et de l’ε–algorithme. Calcolo 12, 317–360 (1975)

    MathSciNet  MATH  Google Scholar 

  27. Brezinski, C.: Forme confluente de l’ε–algorithme topologique. Numer. Math. 23, 363–370 (1975)

    MathSciNet  MATH  Google Scholar 

  28. Brezinski, C.: Accélération de la Convergence en Analyse Numérique, Lecture Notes in Mathematics, vol. 584. Springer–Verlag, Berlin – Heildelberg (1977)

    Google Scholar 

  29. Brezinski, C.: Sur le calcul de certains rapports de déterminants. In: Wuytack, L. (ed.) Padé Approximation and its Applications, Lecture Notes in Mathematics, vol. 765, pp. 184–210. Springer–Verlag, Heidelberg (1979)

  30. Brezinski, C.: Padé–Type Approximation and General Orthogonal Polynomials. International Series of Numerical Mathematics, vol. 50. Basel, Birkhäuser–Verlag (1980)

    Google Scholar 

  31. Brezinski, C.: A general extrapolation algorithm. Numer. Math. 35, 175–187 (1980)

    MathSciNet  MATH  Google Scholar 

  32. Brezinski, C.: About Henrici’s method for nonlinear equations. Symposium on Numerical Analysis and Computational Complex Analysis, Zürich, unpublished (1983)

  33. Brezinski, C.: Recursive interpolation, extrapolation and projection. J. Comput. Appl. Math. 9, 369–376 (1983)

    MathSciNet  MATH  Google Scholar 

  34. Brezinski, C.: Some determinantal identities in a vector space, with applications. In: Werner, H., Bünger, H. J. (eds.) Padé Approximation and its Applications. Bad–Honnef 1983, Lecture Notes in Mathematics, vol. 1071, pp. 1–11. Springer–Verlag, Heidelberg (1984)

  35. Brezinski, C.: The birth and early developments of Padé approximants. In: Rassias, G. M. (ed.) Differential Geometry, Calculus of Variations, and their Applications, pp. 105–121. Marcel Dekker, New York (1985)

  36. Brezinski, C.: Other manifestations of the Schur complement. Linear Alg. Appl. 111, 231–247 (1988)

    MathSciNet  MATH  Google Scholar 

  37. Brezinski, C.: History of Continued Fractions and Padé Approximants. Springer–Verlag, Berlin (1990)

    MATH  Google Scholar 

  38. Brezinski, C.: A Bibliography on Continued Fractions, Padé Approximation, Sequence Transformation, and Related Subjects. Prensas Universitarias de Zaragoza, Zaragoza (1991)

    Google Scholar 

  39. Brezinski, C.: Biorthogonality and its Applications to Numerical Analysis. Marcel Dekker, New York (1992)

    MATH  Google Scholar 

  40. Brezinski, C.: The generalizations of Newton’s interpolation formula due to Mühlbach and Andoyer. Electr. Trans. Numer. Anal. 2, 130–137 (1994)

    MATH  Google Scholar 

  41. Brezinski, C.: Projection Methods for Systems of Equations. North–Holland, Amsterdam (1997)

    MATH  Google Scholar 

  42. Brezinski, C.: Convergence acceleration during the 20th century. J. Comput. Appl. Math. 122, 1–21 (2000)

    MathSciNet  MATH  Google Scholar 

  43. Brezinski, C.: Computational Aspects of Linear Control. Kluwer, Dordrecht (2002)

    MATH  Google Scholar 

  44. Brezinski, C.: Schur complements and applications in numerical analysis. In: Zhang, F. (ed.) The Schur Complement and its Applications, pp. 227–258. Springer, New York (2005)

  45. Brezinski, C.: Some pioneers of extrapolation methods. In: Bultheel, A., Cools, R. (eds.) The Birth of Numerical Analysis, pp. 1–22. World Scientific Publication Co., Singapore (2009)

  46. Brezinski, C.: Cross rules and non–Abelian lattice equations for the discrete and confluent non–scalar ε–algorithms. J. Phys. A: Math. Theor. 43, 205201 (2010)

    MathSciNet  MATH  Google Scholar 

  47. Brezinski, C.: Reminiscences of Peter Wynn, Numer. Algorithms, this issue

  48. Brezinski, C., Chehab, J.-P.: Nonlinear hybrid procedures and fixed point iterations. Numer. Funct. Anal. Optim. 19, 465–487 (1998)

    MathSciNet  MATH  Google Scholar 

  49. Brezinski, C., Chehab, J.–P.: Multiparameter iterative schemes for the solution of systems of linear and nonlinear equations. SIAM J. Sci. Comput. 20, 2140–2159 (1999)

    MathSciNet  MATH  Google Scholar 

  50. Brezinski, C., Crouzeix, M.: Remarques sur le procédé Δ2 d’Aitken. C. R. Acad. Sci. Paris 270 A, 896–898 (1970)

    MATH  Google Scholar 

  51. Brezinski, C., He, Y., Hu, X.-B., Redivo–Zaglia, M., Sun, J.-Q.: Multistep epsilon–algorithm, Shanks’ transformation, and the Lotka–Volterra system by Hirota’s method. Math Comp. 81, 1527–1549 (2012)

    MathSciNet  MATH  Google Scholar 

  52. Brezinski, C., He, Y., Hu, X.-B., Sun, J.-Q.: Cross rules of some extrapolation algorithms. Inverse Probl. 26, 095013 (2010)

    MathSciNet  MATH  Google Scholar 

  53. Brezinski, C., He, Y., Hu, X.-B., Sun, J.-Q., Tam, H.-W.: Confluent form of the multistep ε–algorithm, and the relevant integrable system. Stud. Appl. Math. 127, 191–209 (2011)

    MathSciNet  MATH  Google Scholar 

  54. Brezinski, C., Lembarki, A.: Acceleration of extended Fibonacci sequences. Appl. Numer. Math. 2, 1–8 (1986)

    MathSciNet  MATH  Google Scholar 

  55. Brezinski, C., Redivo-Zaglia, M.: Extrapolation Methods. Theory and Practice. North–Holland, Amsterdam (1991)

    MATH  Google Scholar 

  56. Brezinski, C., Redivo–Zaglia, M.: The simplified topological ε–algorithms for accelerating sequences in a vector space. SIAM J. Sci. Comput. 36, A2227–A2247 (2014)

    MathSciNet  MATH  Google Scholar 

  57. Brezinski, C., Redivo–Zaglia, M.: The simplified topological ε–algorithms: software and applications. Numer. Algorithms 74, 1237–1260 (2017)

    MathSciNet  MATH  Google Scholar 

  58. Brezinski, C., Redivo–Zaglia, M.: Shanks function transformations in a vector space. Appl. Numer. Math. 116, 57–63 (2017)

    MathSciNet  MATH  Google Scholar 

  59. Brezinski, C., Redivo–Zaglia, M.: Hirota’s bilinear method and the ε–algorithm, Rev. Roumaine Math. Pures Appl., Tome LXIV, No. 4/2018, to appear

  60. Brezinski, C., Redivo–Zaglia, M., Saad, Y.: Shanks sequence transformations and Anderson acceleration, SIAM Rev. 60, 646–669 (2018)

  61. Brezinski, C., Rieu, A.C.: The solution of systems of equations using the vector ε–algorithm, and an application to boundary value problems. Math. Comp. 28, 731–741 (1974)

    MATH  Google Scholar 

  62. Brezinski, C., Sadok, H.: Vector sequence transformations and fixed point methods. In: Taylor, C., et al. (eds.) Numerical Methods in Laminar and Turbulent Flows, pp. 3–11. Pineridge Press, Swansea (1987)

  63. Brezinski, C., Van Iseghem, J.: Padé approximations. In: Ciarlet, P.G., Lions, J.L. (eds.) Handbook of Numerical Analysis, vol. III, pp. 47–222, North–Holland (1994)

  64. Brezinski, C., Walz, G.: Sequences of transformations and triangular recursion schemes, with applications in numerical analysis. J. Comput. Appl. Math. 34, 361–383 (1991)

    MathSciNet  MATH  Google Scholar 

  65. Brezinski, C., Wuytack, L.: Numerical analysis in the twentieth century. In: Brezinski, C., Wuytack, L. (eds.) Numerical Analysis: Historical Developments in the 20th Century, pp. 1–40, North–Holland (2001)

  66. Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comp. 19, 577–593 (1965)

    MathSciNet  MATH  Google Scholar 

  67. Buhmann, M.D., Fletcher, R.: M.J.D. Powell’s contributions to numerical mathematics. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization. Tributes to M.J.D. Powell, pp. 1–30. Cambridge University Press, Cambridge (1997)

  68. Cabay, S., Jackson, L.W.: A polynomial extrapolation method for finding limits and antilimits of vector sequences. SIAM J. Numer. Anal. 13, 734–752 (1976)

    MathSciNet  MATH  Google Scholar 

  69. Cantor, C.: Variants of the Secant Method for Solving Nonlinear Systems of Equations. Report X–733–71–48, Goddard Space Flight Center, Greenbelt (1971)

    Google Scholar 

  70. Cantor, C., Emad, F.P.: A New Iteration for Locating Equilibrium Points in Nonlinear Systems. Report, X–732–69–306, Goddard Space Flight Center, Greenbelt (1969)

    Google Scholar 

  71. Capehart, S.R.: Techniques for Accelerating Iterative Methods for the Solution of Mathematical Problems. Ph.D. Thesis, Oklahoma State University, Stillwater (1989)

    Google Scholar 

  72. Cauchy, A.L.: Cours d’Analyse de l’École Royale Polytechnique. Première Partie: Analyse Algébrique. Imprimerie Royale, Paris (1821)

    MATH  Google Scholar 

  73. Chabert, J.-L., et al.: A History of Algorithms. Springer–Verlag, Berlin (1999)

    MATH  Google Scholar 

  74. Chang, X.-K., He, Y., Hu, X.-B., Li, S.-H.: A new integrable convergence acceleration algorithm for computing Brezinski–Durbin–Redivo–Zaglia’s sequence transformation via pfaffians. Numer. Algorithms 78, 87–106 (2018)

    MathSciNet  MATH  Google Scholar 

  75. Chatelin, F., Miranker, W.L.: Acceleration by aggregation of successive approximation methods. Linear Algebra Appl. 43, 17–47 (1982)

    MathSciNet  MATH  Google Scholar 

  76. Chiò, F.: Mémoire sur les Fonctions Connues sous le Nom des Résultantes ou de Déterminans. A Pons et C., Turin (1853)

    Google Scholar 

  77. Cipolla, S., Redivo–Zaglia, M., Tudisco, F.: Extrapolation methods for fixed point multilinear PageRank computations, submitted

  78. Coope, I.D., Graves–Morris, P.R.: The rise and fall of the vector epsilon algorithm. Numer. Algorithms 5, 275–286 (1993)

    MathSciNet  MATH  Google Scholar 

  79. Cordellier, F.: Interprétation Géométrique de l’ε–Algorithme, Internal Report, Laboratoire d’Analyse Numérique et d’Optimisation, Université des Sciences et Techniques de Lille (1973)

  80. Cordellier, F.: Particular rules for the vector ε–algorithm. Numer. Math. 27, 203–207 (1977)

    MathSciNet  MATH  Google Scholar 

  81. Cordellier, F.: Démonstration algébrique de l’extension de l’identité de Wynn aux tables de Padé non normales. In: Wuytack, L. (ed.) Padé Approximation and its Applications, Lecture Notes in Mathematics, vol. 765, pp. 36–60. Springer–Verlag, Berlin (1979)

  82. Cordellier, F.: Utilisation de l’invariance homographique dans les algorithmes de losange. In: Werner, H., Bünger, H.J. (eds.) Padé Approximation and its Applications Bad Honnef 1983, Lecture Notes in Mathematics, vol. 1071, pp. 62–94. Springer–Verlag, Berlin (1984)

  83. Cordellier, F.: Interpolation Rationnelle et autres Questions: Aspects Algorithmiques et Numériques, Thèse de Doctorat d’État ès Sciences Mathématiques, Université des Sciences et Techniques de Lille (1989)

  84. Cuyt, A.: The epsilon–algorithm and multivariate Padé approximants. Numer. Math. 40, 39–46 (1982)

    MathSciNet  MATH  Google Scholar 

  85. Cuyt, A.: The epsilon–algorithm and Padé approximants in operator theory. SIAM J. Math. Anal. 14, 1009–1014 (1983)

    MathSciNet  MATH  Google Scholar 

  86. Davidon, W.: Variable Metric Methods for Minimization, A.E.C. Res. and Develop. Rept. ANL–5990, Argonne National Laboratory, Argonne, Illinois (1959)

  87. Davis, P.J.: Interpolation and Approximation. Blaisdell, New York (1961)

    Google Scholar 

  88. Delahaye, J.P.: Sequence Transformations. Springer–Verlag, Berlin (1988)

    MATH  Google Scholar 

  89. Delahaye, J.P., Germain–Bonne, B.: Résultats négatifs en accélération de la convergence. Numer. Math. 35, 443–457 (1980)

    MathSciNet  MATH  Google Scholar 

  90. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its applications to quasi–Newton methods. Math. Comp. 28, 549–560 (1974)

    MathSciNet  MATH  Google Scholar 

  91. Dodgson, C.L.: Condensation of determinants, being a new and brief method for computing their arithmetical values. Proc. Roy. Soc. Lond. 15, 150–155 (1866)

    Google Scholar 

  92. Draux, A.: The epsilon algorithm in a non–commutative algebra. J. Comput. Appl. Math. 19, 9–21 (1987)

    MathSciNet  MATH  Google Scholar 

  93. Eddy, R.P.: Extrapolation to the limit of a vector sequence. In: Wang, P.C.C. (ed.) Information Linkage between Applied Mathematics and Industry, pp. 387–396. Academic Press, New York (1979)

  94. Eddy, R.P.: The Even–Rho and Even–Epsilon Algorithms for Accelerating Convergence of a Numerical Sequence, Report DTNSRDC–81/083, David W. Taylor Naval Ship Research and Development Center, Bethesda, Maryland (1981)

  95. Eddy, R.P.: Acceleration of Convergence of a Vector Sequence by Reduced Rank Extrapolation, Report DTNSRDC–81/084, David W. Taylor Naval Ship Research and Development Center, Bethesda, Maryland (1981)

  96. Fang, H., Saad, Y.: Two classes of multisecant methods for nonlinear acceleration. Numer. Linear Algebra Appl. 16, 197–221 (2009)

    MathSciNet  MATH  Google Scholar 

  97. Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Watson, G.A., (ed.) Numerical Analysis, Lecture Notes in Mathematics, vol. 506, pp. 73–89. Springer–Verlag, Berlin (1976)

  98. Fletcher, R., Powell, M.J.D.: A rapidly convergent descent method for minimization. Comput. J. 6, 163–168 (1963)

    MathSciNet  MATH  Google Scholar 

  99. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    MathSciNet  MATH  Google Scholar 

  100. Ford, W.F., Sidi, A.: An algorithm for a generalization of the Richardson extrapolation process. SIAM J. Numer. Anal. 24, 1212–1232 (1987)

    MathSciNet  MATH  Google Scholar 

  101. Ford, W.F., Sidi, A.: Recursive algorithms for vector extrapolation methods. Appl. Numer. Math. 4, 477–489 (1988)

    MathSciNet  MATH  Google Scholar 

  102. Forsythe, G.E.: Solving linear algebraic equations can be interesting. Bull. Amer. Math. Soc. 59, 299–329 (1953)

    MathSciNet  MATH  Google Scholar 

  103. Frobenius, G.: Über Relationen zwischen den Näherungsbrüchen von Potenzreihen. J. Reine Angew. Math. 90, 1–17 (1881)

    MathSciNet  MATH  Google Scholar 

  104. Fürstenau, E.: Darstellung der reellen Wurzeln algebraischer Gleichungen durch Determinanten der Coefficienten. N.G Elwert’sche Universitäts–Buchdruckerei, Marburg (1860)

    Google Scholar 

  105. Fürstenau, E.: Neue Methode zur Darstellung und Berechnung der imaginären Wurzeln algebraischer Gleichungen durch Determinanten der Coeffizienten. Programmabhandlung, Marburg (1867)

    Google Scholar 

  106. Gander, W., Gander, M.J., Kwok, F.: Scientific Computing. An Introduction using Maple and MATLAB. Springer, Cham (2014)

    MATH  Google Scholar 

  107. Gander, W., Golub, G.H., Gruntz, D.: Solving linear equations by extrapolation. In: Kowalik, J.S (ed.) Supercomputing, NATO ASI Series (Series F: Computer and Systems Sciences), vol. 62, pp. 279–293. Springer, Berlin (1990)

  108. Gauss, C.F.: Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. F. Perthes and J. H. Besser, Hamburg (1809)

    MATH  Google Scholar 

  109. Gekeler, E.: On the solution of systems of equations by the epsilon algorithm of Wynn. Math. Comp. 26, 427–436 (1972)

    MathSciNet  MATH  Google Scholar 

  110. Germain–Bonne, B.: Transformations de suites, Revue française d’automatique, informatique, recherche opérationnelle. Mathématique. R.A.I.R.O. 7, 84–90 (1973)

    MATH  Google Scholar 

  111. Germain–Bonne, B.: Accélération de la Convergence par Projection, Internal Report, Laboratoire d’Analyse Numérique et d’Optimisation, Université des Sciences et Techniques de Lille (1975)

  112. Germain–Bonne, B.: Estimation de la Limite de Suites et Formalisation de Procédés d’Accélération de la Convergence, Thèse de Doctorat d’État ès Sciences Mathématiques, Université des Sciences et Techniques de Lille (1978)

  113. Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)

    MATH  Google Scholar 

  114. Golub, G.H., O’Leary, D.P.: Some history of the conjugate gradient and Lanczos methods. SIAM Rev. 31, 50–102 (1989)

    MathSciNet  MATH  Google Scholar 

  115. Gragg, W.B.: The Padé table and its relation to certain algorithms of numerical analysis. SIAM Rev. 14, 1–62 (1972)

    MathSciNet  MATH  Google Scholar 

  116. Graves–Morris, P.R.: Vector valued rational interpolants I. Numer. Math. 42, 331–348 (1983)

    MathSciNet  MATH  Google Scholar 

  117. Graves–Morris, P.R.: Extrapolation methods for vector sequences. Numer. Math. 61, 475–487 (1992)

    MathSciNet  MATH  Google Scholar 

  118. Graves–Morris, P.R.: A new approach to acceleration of convergence of a sequence of vectors. Numer. Algorithms 11, 189–201 (1996)

    MathSciNet  MATH  Google Scholar 

  119. Graves–Morris, P.R., Jenkins, C.D.: Generalised inverse vector–valued rational interpolation. In: Werner, H., Bünger, H.J. (eds.) Padé Approximation and its Applications, Lecture Notes in Mathematics, vol. 1071, pp. 144–156. Springer, Berlin (1984)

  120. Graves–Morris, P.R., Jenkins, C.D.: Vector–valued rational interpolants III. Constr. Approx. 2, 263—289 (1986)

    MathSciNet  MATH  Google Scholar 

  121. Graves–Morris, P.R., Roberts, D.E.: From matrix to vector Padé approximants. J. Comput. Appl. Math. 51, 205–236 (1994)

    MathSciNet  MATH  Google Scholar 

  122. Graves–Morris, P.R., Roberts, D.E., Salam, A.: The epsilon algorithm and related topics. J. Comput. Appl. Math. 122, 51–80 (2000)

    MathSciNet  MATH  Google Scholar 

  123. Gray, H.L., Atchison, T.A., McWilliams, G.V.: Higher order G–transformations. SIAM J. Numer. Anal. 3, 365–381 (1971)

    MathSciNet  MATH  Google Scholar 

  124. Greville, T.N.E.: On Some Conjectures of P. Wynn Concerning the ε–Algorithm, MRC Technical Summary Report # 877, Madison, Wisconsin (1968)

  125. Gutknecht, M.H., Parlett, B.N.: From qd to L R, or, how were the qd and L R algorithms discovered?. IMA J. Numer. Anal. 31, 741–754 (2011)

    MathSciNet  MATH  Google Scholar 

  126. Gutzler, C.H.: An Iterative Method of Wegstein for Solving Simultaneous Nonlinear Equations. Master Thesis, Oregon State College (1959)

  127. Hadamard, J.: Éssai sur l’Étude des Fonctions données par leur Développement de Taylor, Thèse de Doctorat d’État ès Sciences Mathématiques, Université de Paris; also in J. Math. Pures Appl., 4è, sér. 8, pp. 101–186 (1892)

  128. Hadamard, J., Mandelbrojt, S.: La Série de Taylor et son Prolongement Analytique, Scientia 41, Paris (1926)

  129. Haelterman, R., Degroote, J., Heule, D.V., Vierendeels, J.: On the similarities between the quasi–Newton inverse least squares method and GMRES. SIAM J. Numer. Anal. 47, 4660–4679 (2010)

    MathSciNet  MATH  Google Scholar 

  130. Håvie, T.: Generalized Neville type extrapolation schemes. BIT 19, 204–213 (1979)

    MathSciNet  MATH  Google Scholar 

  131. Hankel, H.: Über eine besondere Classe des symmetrischen Determinanten. Inaugural Dissertation, Universität Göttingen (1861)

  132. He, Y., Hu, X.-B., Sun, J.-Q., Weniger, E.J.: Convergence acceleration algorithm via the lattice Boussinesq equation. SIAM J. Sci. Comput. 33, 1234–1245 (2011)

    MathSciNet  MATH  Google Scholar 

  133. Henrici, P.: The quotient–difference–algorithm. Natl. Bur. Stand. Appl. Math. Ser. 49, 23–46 (1958)

    MathSciNet  MATH  Google Scholar 

  134. Henrici, P.: Elements of Numerical Analysis. Wiley, New York (1964)

    MATH  Google Scholar 

  135. Henrici, P.: Applied and Computational Complex Analysis, vol. 1. Wiley, New York (1974)

    MATH  Google Scholar 

  136. Henrici, P.: Essentials of Numerical Analysis with Pocket Calculator Demonstrations. Wiley, New York (1982)

    MATH  Google Scholar 

  137. Henrici, P.: Essentials of Numerical Analysis with Pocket Calculator Demonstrations, Solutions Manual. Wiley, New York (1982)

    MATH  Google Scholar 

  138. Hirayama, A., Shimodaira, K., Hirose, H.: Takakazu Seki’s Collected Works Edited with Explanations. Osaka Kyoiku Tosho, Osaka (1974)

    Google Scholar 

  139. Hirota, R.: Exact solution of the Korteweg–de Vries equation for multiple collisions of solitons. Phys. Rev. Lett. 27, 1192–1194 (1971)

    MATH  Google Scholar 

  140. Hirota, R.: The Direct Method in Soliton Theory. Cambridge University Press, Cambridge (1992)

    Google Scholar 

  141. Holme, H.: Beitrag zur Berechnung des effektiven Zinsfusses bei Anleihen. Skand. Aktuarietidskr. 15, 225–250 (1932)

    MATH  Google Scholar 

  142. Jacobi, C.G.J.: Über die Darstellung einer Reihe gegebener Werthe durch eine gebrochene rationale Function. J. Reine Angew. Math. 30, 127–156 (1846). Gesammelte Werke, Tome 3, Reiner, Berlin, pp. 479–511 (1882–1891)

    MathSciNet  Google Scholar 

  143. Jamieson, M.J., O’Beirne, T.H.: A note on the generalisation of Aitken δ 2 transformation. J. Phys. B: Atom. Molec. Phys. 11, L31–L35 (1978)

    MathSciNet  Google Scholar 

  144. Jbilou, K.: Méthodes d’Extrapolation et de Projection. Applications aux Suites de Vecteurs, Thèse de 3ème cycle, Université des Sciences et Techniques de Lille (1988)

  145. Jbilou, K.: A general projection algorithm for solving systems of linear equations. Numer. Algorithms 4, 361–377 (1993)

    MathSciNet  MATH  Google Scholar 

  146. Jbilou, K., Messaoudi, A.: Matrix recursive interpolation algorithm for block linear systems. Linear Algebra Appl. 294, 137–154 (1999)

    MathSciNet  MATH  Google Scholar 

  147. Jbilou, K., Messaoudi, A.: Block extrapolation methods with applications. Appl. Numer. Math. 106, 154–164 (2016)

    MathSciNet  MATH  Google Scholar 

  148. Jbilou, K., Messaoudi, A., Tabaâ, K.: Some Schur complement identities and applications to matrix extrapolation methods. Linear Algebra Appl. 392, 195–210 (2004)

    MathSciNet  MATH  Google Scholar 

  149. Jbilou, K., Sadok, H.: Some results about vector extrapolation methods and related fixed point iteration. J. Comp. Appl. Math. 36, 385–398 (1991)

    MathSciNet  MATH  Google Scholar 

  150. Jbilou, K., Sadok, H.: Analysis of some vector extrapolation methods for solving systems of linear equations. Numer. Math. 70, 73–89 (1995)

    MathSciNet  MATH  Google Scholar 

  151. Jbilou, K., Sadok, H.: LU–implementation of the modified minimal polynomial extrapolation method. IMA J. Numer. Anal. 19, 549–561 (1999)

    MathSciNet  MATH  Google Scholar 

  152. Jbilou, K., Sadok, H.: Vector extrapolation methods. Applications and numerical comparison. J. Comput. Appl. Math. 122, 149–165 (2000)

    MathSciNet  MATH  Google Scholar 

  153. Jbilou, K., Sadok, H.: Matrix polynomial and epsilon–type extrapolation methods with applications. Numer. Algorithms 68, 107–119 (2015)

    MathSciNet  MATH  Google Scholar 

  154. Jennings, A.: Accelerating the convergence of matrix iterative processes. J. Inst. Maths Appl. 8, 99–110 (1971)

    MathSciNet  MATH  Google Scholar 

  155. Joyce, D.C.: Survey of extrapolation processes in numerical analysis. SIAM Rev. 13, 435–490 (1971)

    MathSciNet  MATH  Google Scholar 

  156. Kaniel, S.: Estimates for some computational techniques in linear algebra. Math. Comp. 20, 369–378 (1966)

    MathSciNet  MATH  Google Scholar 

  157. Kaniel, S., Stein, J.: Least–square acceleration of iterative methods for linear equations. J. Optim. Theory Appl. 14, 431–437 (1974)

    MathSciNet  MATH  Google Scholar 

  158. Karapiperi, A., Redivo–Zaglia, M., Russo, M.R.: Generalizations of Sylvester’s determinantal identity, arXiv:http://arXiv.org/abs/1503.00519

  159. König, J.: Über eine Eigenschaft der Potenzreihen. Math. Ann. 23, 447–449 (1884)

    MathSciNet  MATH  Google Scholar 

  160. Krasnosel’skii, M.A., Krein, S.G.: An iterative process with minimum discrepancies. Matem. Sb. 31(73), 315–334 (1952)

    Google Scholar 

  161. Kummer, E.E.: Über die Convergenz und Divergenz der unendlichen Reihen. J. Reine Angew. Math. 13, 171–184 (1835)

    MathSciNet  Google Scholar 

  162. Kummer, E.E.: Eine neue Methode, die numerische Summen langsam convergirender Reihen zu berechnen. J. Reine Angew. Math. 16, 206–214 (1837)

    MathSciNet  Google Scholar 

  163. Lambert, D.: The Atom of Universe : the Life and Work of Georges Lemaître. Copernicus Center Press, Kraków (2014)

    Google Scholar 

  164. Lanczos, C.: Solution of systems of linear equations by minimized iterations. J. Res. Natl. Bur. Stand. 49, 33–53 (1952)

    MathSciNet  Google Scholar 

  165. Le Ferrand, H.: The quadratic convergence of the topological epsilon algorithm for systems of nonlinear equations. Numer. Algorithms 3, 273–284 (1992)

    MathSciNet  MATH  Google Scholar 

  166. Le Ferrand, H.: Genèse et diffusion d’un théorème de Robert de Montessus de Ballore sur les fractions continues algébriques, prépublication HAL, https://hal.archives-ouvertes.fr/hal-00521135v3 (2014)

  167. Le Ferrand, H.: The rational iteration method by Georges Lemaître, Numer. Algorithms, this issue and, in French, https://hal.archives-ouvertes.fr/hal-01176009(2015)

  168. Le Ferrand, H.: Robert de Montessus de Ballore’s 1902 theorem on algebraic continued fractions : genesis and circulation, arXiv:1307.3669

  169. Le Ferrand, H.: 1902, un théorème pour la postérité?, Images des Mathématiques, CNRS. http://images.math.cnrs.fr/1902-un-theoreme-pour-la-posterite.html (2017)

  170. Ledermann, W.: Obituary, A.C. Aitken, DSc, FRS. Proc. Edinb. Math. Soc. 16, 151–176 (1968)

    Google Scholar 

  171. Lemaître, G.: L’itération rationnelle. Acad. Belgique, Bull. Cl. Sci. (5) 28, 347–354 (1942)

    MathSciNet  MATH  Google Scholar 

  172. Lemaître, G.: Intégration d’une équation différentielle par itération rationnelle. Acad. Belgique, Bull. Cl. Sci. (5) 28, 815–825 (1942)

    MathSciNet  MATH  Google Scholar 

  173. Lemaréchal, C.: Une méthode de résolution de certains systèmes non linéaires bien posés. C.R. Acad. Sci. Paris, sér. A 272, 605–607 (1971)

    MathSciNet  MATH  Google Scholar 

  174. López Lagomasino, G., Zaldívar Gerpe, Y.: Higher order recurrences and row sequences of Hermite–Padé, approximation, arXiv:1801.02650v1.8 (2018)

  175. Lubkin, S.: A method of summing infinite series. J. Res. Natl. Bur. Stand. 48, 228–254 (1952)

    MathSciNet  Google Scholar 

  176. Ludwig, R.: Verbesserung einer Iterationsfolge bei Gleichungssystemen. Z. Angew. Math. Mech. 32, 232–234 (1952)

    MATH  Google Scholar 

  177. Mandelbrojt, S.: Les Singularités des Fonctions Analytiques représentées par une Série de Taylor, Mémorial des Sciences Mathématiques, Fascicule 54, Gauthier–Villars, Paris (1932)

  178. Marx, I.: Remark concerning a non–linear sequence–to–sequence transform. J. Math. Phys. 42, 334–335 (1963)

    MathSciNet  MATH  Google Scholar 

  179. Maxwell, J.C.: A Treatise on Electricity and Magnetism. Clarendon Press, Oxford (1873)

    MATH  Google Scholar 

  180. Maz’ya, V., Shaposhnikova, T.: Jacques Hadamard, a Universal Mathematician. American Mathematical Society and London Mathematical Society, Providence and London (1998)

    MATH  Google Scholar 

  181. McLeod, J.B.: A note on the ε–algorithm. Computing 7, 17–24 (1971)

    MathSciNet  Google Scholar 

  182. McWilliams, G.V.: The e m and Higher Order G Transform. Ph.D. thesis, Texas Technological College, Lubbock (1969)

    Google Scholar 

  183. Meidell, B.: Betrachtungen über den effektiven Zinsfuss bei Anleihen. Skand. Aktuarietidskr. 15, 159–174 (1932)

    MATH  Google Scholar 

  184. Meinardus, G., Taylor, G.D.: Lower estimates for the error of the best uniform approximation. J. Approx. Theory 16, 150–161 (1976)

    MathSciNet  MATH  Google Scholar 

  185. Mešina, M.: Convergence acceleration for the iterative solution of x = A x + f. Comput. Methods Appl. Mech. Eng. 10, 165–173 (1977)

    MathSciNet  MATH  Google Scholar 

  186. Mešina, M., Emendörfer, D.: Transmission probability method for neutron transport calculations in non–uniform reactor lattices. Atomkernenergie 26, 163–168 (1975)

    Google Scholar 

  187. Messaoudi, A.: Matrix recursive projection and interpolation algorithms. Linear Algebra Appl. 202, 71–89 (1994)

    MathSciNet  MATH  Google Scholar 

  188. Messaoudi, A.: Some properties of the recursive projection and interpolation algorithms. IMA J. Numer. Anal. 15, 307–318 (1995)

    MathSciNet  MATH  Google Scholar 

  189. Messaoudi, A.: Recursive interpolation algorithm: a formalism for solving systems of linear equations–I. Direct methods. J. Comput. Appl. Math. 76, 13–30 (1996)

    MathSciNet  MATH  Google Scholar 

  190. Messaoudi, A.: Recursive interpolation algorithm: a formalism for solving systems of linear equations–II. Iterative methods. J. Comput. Appl. Math. 76, 31–53 (1996)

    MathSciNet  MATH  Google Scholar 

  191. Messaoudi, A.: Matrix extrapolation algorithms. Linear Algebra Appl. 256, 49–73 (1997)

    MathSciNet  MATH  Google Scholar 

  192. Messaoudi, A., Errachid, M., Jbilou, K., Sadok, H.: GRPIA: a new algorithm for computing interpolation polynomials, Numer. Algorithms, this issue

  193. Messaoudi, A., Sadok, H.: Recursive polynomial interpolation algorithm (RPIA). Numer. Algorithms 76, 675–694 (2017)

    MathSciNet  MATH  Google Scholar 

  194. Meurant, G.: Computer Solution of Large Linear Systems. North–Holland, Amsterdam (1999)

    MATH  Google Scholar 

  195. Miellou, J.C.: Extrapolation aggregation algorithm of monotone kind. Application to “one obstacle’s” stationnary problems. In: Free Boundary Problems, Pavia, Sept.–Oct. 1979, vol. n, pp. 411–438. Istituto Nazionale di Alta Matematica Francesco Severi, Roma (1980)

  196. Minesaki, Y., Nakamura, Y.: The discrete relativistic Toda molecule equation and a Padé approximation algorithm. Numer. Algorithms 27, 219–235 (2001)

    MathSciNet  MATH  Google Scholar 

  197. Miranker, W.L., Pan, V.: Methods of aggregation. Linear Algebra Appl. 29, 231–257 (1980)

    MathSciNet  MATH  Google Scholar 

  198. de Montessus de Ballore, R.: Sur les fractions continues algébriques. Bull. Soc. Math. France 30, 28–36 (1902)

    MathSciNet  MATH  Google Scholar 

  199. de Montessus de Ballore, R.: Sur les Fractions Continues Algébriques, Thèse de Doctorat d’État ès Sciences Mathématiques, Faculté des Sciences de Paris (1905)

  200. Mühlbach, G.: Neville–Aitken algorithms for interpolating by functions of Čebyšev–systems in the sense of Newton and in a generalized sense of Hermite. In: Law, A.G., Sahney, B.N. (eds.) Theory of Approximation with Applications, pp. 200–212. Academic Press, New York (1976)

  201. Nägelsbach, H.: Studien zu Fürstenau’s neuer Methode der Darstellung und Berechnung der Wurzeln algebraischer Gleichungen durch Determinanten der Coefficienten. Arch. Math. Phys. 59, 147–192 (1876); 61, 19–85 (1877)

  202. Nagai, A., Satsuma, J.: Discrete soliton equations and convergence acceleration algorithms. Phys. Lett. A 209, 305–312 (1995)

    MathSciNet  MATH  Google Scholar 

  203. Nagai, A., Tokihiro, T., Satsuma, J.: The Toda molecule equation and the ε–algorithm. Math. Comp. 67, 1565–1575 (1998)

    MathSciNet  MATH  Google Scholar 

  204. Nakamura, Y.: A new approach to numerical algorithms in terms of integrable systems. In: Ibaraki, T., Inui, T., Tanaka, K. (eds.) Proceedings of International Conference on Informatics Research for Development of Knowledge Society Infrastructure ICKS 2004, pp. 194–205. IEEE Computer Society Press (2004)

  205. Ni, P.: Anderson Acceleration of Fixed–Point Iteration with Applications to Electronic Structure Computations. Ph.D. thesis, Worcester Polytechnic Institute, Worcester (2009)

    Google Scholar 

  206. Nievergelt, Y.: Aitken’s and Steffensen’s accelerations in several variables. Numer. Math. 59, 295–310 (1991)

    MathSciNet  MATH  Google Scholar 

  207. Noda, T.: The Aitken–Steffensen formula for systems of nonlinear equations. Sûgaku 33, 369–372 (1981). II: Sûgaku 38, 83–85 (1986) (in Japanese)

    MathSciNet  MATH  Google Scholar 

  208. Noda, T.: The Aitken–Steffensen iteration method for systems of nonlinear equations. Proc Japan Acad. 60A, 18–21 (1984); III: 62A, 174–177 (1986); IV: 65A, 260–263 (1990); V: 68A, 37–40 (1992)

  209. Nörlund, N. E.: Fractions continues et différences réciproques. Acta Math. 34, 1–108 (1911)

    MathSciNet  MATH  Google Scholar 

  210. O’Beirne, T.H.: On Linear Iterative Processes and on Methods of Improving the Convergence of Certain Types of Iterated Sequences, Technical report S.T.R. 491. Torpedo Experimental Establishment, Greenock (1947)

    Google Scholar 

  211. Ogborn, M.E.: Johan Frederik Steffensen. J. Inst. Actuar. 88, 251–253 (1962)

    Google Scholar 

  212. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, San Diego (1970)

    MATH  Google Scholar 

  213. Osada, N.: The early history of convergence acceleration methods. Numer. Algorithms 60, 205–221 (2012)

    MathSciNet  MATH  Google Scholar 

  214. Padé, H.: Sur la représentation approchée d’une fonction par des fractions rationnelles. Ann. Sci. Éc. Norm. Supér. (3) 9, 3–93 (1892)

    MathSciNet  MATH  Google Scholar 

  215. Padé, H.: Œuvres. In: Brezinski, C. (ed.) . Libraire Scientifique et Technique Albert Blanchard, Paris (1984)

  216. Paige, C.C.: The Computation of Eigenvalues and Eigenvectors of Very Large Sparse Matrices, Ph.D. Thesis, University of London (1971)

  217. Papageorgiou, V., Grammaticos, B., Ramani, A.: Integrable lattices and convergence acceleration algorithms. Phys. Lett. A A179, 111–115 (1993)

    MathSciNet  MATH  Google Scholar 

  218. Papageorgiou, V., Grammaticos, B., Ramani, A.: Integrable difference equations and numerical analysis algorithms. In: Levi, D., Vinet, L., Winternitz, P. (eds.) Symmetries and Integrability of Difference Equations. Proceedings of the workshop held in Estérel, Québec, May 22–29, 1994, pp. 269–280. CRM Proceedings & Lecture Notes, 9, American Mathematical Society, Providence (1996)

  219. Pennacchi, R.: Le trasformazioni razionali di una successione. Calcolo 5, 37–50 (1968)

    MathSciNet  MATH  Google Scholar 

  220. Poincaré, H.: Sur les équations linéaires aux différentielles ordinaires et aux différences finies. Amer. J. Math. 7, 203–258 (1885)

    MathSciNet  MATH  Google Scholar 

  221. Pople, J.A., Krishnan, R., Schlegel, H.B., Binkley, J.S.: Derivative studies in Hartree–Fock and Møller–Plesset theories. Int. J. Quantum Chem., Quantum, Chem. Symp. 13, 225–241 (1979)

    Google Scholar 

  222. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes: The Art of Scientific Computing. 3rd edn. Cambridge University Press, Cambridge (2007)

  223. Prony, R.: Éssai experimental et analytique: sur les lois de la dilatabilité des fluides élastiques et sur celles de la force expansive de la vapeur de l’eau et de la vapeur de l’alkool, à différentes températures, Journal de l’École Polytechnique, Volume 1, Cahier 22, Floréal et Plairial, An III, pp. 24–76 (1795)

  224. Pugachëv, B. P.: The use of poorly converging iteration processes for the solution of systems of linear equations. Zh. Vychisl. Mat. Mat. Fiz. 8(6), 1318–1321 (1968). USSR Comput. Maths. Math. Phys., 8(6), 172–176 (1977)

    MathSciNet  Google Scholar 

  225. Pugachëv, B. P.: Acceleration of convergence of iterative processes and a method of solving systems of non–linear equations. Zh. Vychisl. Mat. Mat. Fiz. 17 (5), 1302–1308 (1977). USSR Comput. Maths. Math. Phys. 17(5), 199–207 (1977)

    Google Scholar 

  226. Pulay, P.: Convergence acceleration in iterative sequences: the case of SCF iteration. Chem. Phys. Lett. 73, 393–398 (1980)

    Google Scholar 

  227. Pulay, P.: Improved SCF convergence acceleration. J. Comp. Chem. 3, 556–560 (1982)

    Google Scholar 

  228. Pye, W.C., Atchison, T.A.: An algorithm for the computation of higher order G–transformation. SIAM J. Numer. Anal. 10, 1–7 (1973)

    MathSciNet  MATH  Google Scholar 

  229. Pyle, L.D.: A generalized inverse ε–algorithm for constructing intersection projection matrices with applications. Numer. Math. 10, 86–102 (1967)

    MathSciNet  MATH  Google Scholar 

  230. Redivo–Zaglia, M.: Pseudo–Schur complements and their properties. Appl. Numer. Math. 50, 511–519 (2004)

    MathSciNet  MATH  Google Scholar 

  231. Redivo–Zaglia, M., Tudisco, F.: Shifted and extrapolated power sequences for tensor ℓ p–eigenvalues, submitted

  232. Riesz, F.: Les Systèmes d’Équations Linéaires à une Infinité d’Inconnues. Gauthier–Villars, Paris (1952)

    MATH  Google Scholar 

  233. Roberts, D.E.: The vector epsilon algorithm – a residual approach. Numer. Algorithms 29, 209–227 (2002)

    MathSciNet  MATH  Google Scholar 

  234. Rohwedder, T., Schneider, R.: An analysis for the DIIS acceleration method used in quantum chemistry calculations. J. Math. Chem. 49, 1889–1914 (2011)

    MathSciNet  MATH  Google Scholar 

  235. Roland, Ch., Varadhan, R.: New iterative schemes for nonlinear fixed point problems, with applications to problems with bifurcations and incomplete–data problems. Appl. Numer. Math. 55, 215–226 (2005)

    MathSciNet  MATH  Google Scholar 

  236. Rosen, J.B.: The gradient projection method for nonlinear programming. Part I. Linear constraints. J. SIAM 8, 181–217 (1960)

    MATH  Google Scholar 

  237. Rutishauser, H.: Ein kontinuierliches Analogon zum Quotienten–Differenzen–Algorithmus. Arch. Math. (Basel) 5, 132–137 (1954)

    MathSciNet  MATH  Google Scholar 

  238. Rutishauser, H.: Anwendungen des Quotienten–Differenzen–Algorithmus. Z. Angew. Math. Phys. 5, 496–508 (1954)

    MathSciNet  MATH  Google Scholar 

  239. Rutishauser, H.: Der Quotienten–Differenzen-Algorithmus. Basel, Birkhäuser Verlag (1957)

    MATH  Google Scholar 

  240. Rutishauser, H.: Une méthode pour la détermination des valeurs propres d’une matrice. C.R. Acad. Sci. Paris 240, 34–36 (1955)

    MathSciNet  MATH  Google Scholar 

  241. Rutishauser, H., Bauer, F.L.: Détermination des vecteurs propres d’une matrice par une méthode itérative avec convergence quadratique. C.R. Acad. Sci. Paris 240, 1680–1681 (1955)

    MathSciNet  MATH  Google Scholar 

  242. Saad, Y.: On the rates of convergence of the Lanczos and block–Lanczos methods. SIAM J. Numer. Anal. 17, 687–706 (1980)

    MathSciNet  MATH  Google Scholar 

  243. Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996)

    MATH  Google Scholar 

  244. Saad, Y., Schultz, M.H.: GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)

    MathSciNet  MATH  Google Scholar 

  245. Saad, Y., van der Vorst, H.A.: Iterative solution of linear systems in the 20th century. J. Comput. Appl. Math. 123, 1–33 (2000)

    MathSciNet  MATH  Google Scholar 

  246. Sadok, H.: About Henrici’s transformation for accelerating vector sequences. J. Comput. Appl. Math. 29, 101–110 (1990)

    MathSciNet  MATH  Google Scholar 

  247. Sadok, H.: Quasilinear vector extrapolation methods. Linear Algebra Appl. 190, 71–85 (1993)

    MathSciNet  MATH  Google Scholar 

  248. Salam, A.: Non–commutative extrapolation algorithms. Numer. Algorithms 7, 225–251 (1994)

    MathSciNet  MATH  Google Scholar 

  249. Salam, A.: On the vector–valued Padé approximants and the vector ε–algorithm. In: Cuyt, A. (ed.) Nonlinear Numerical Methods and Rational Approximation II, pp. 291–301. Kluwer Academic Publishers, Dordrecht (1994)

  250. Salam, A.: An algebraic approach to the vector ε–algorithm. Numer. Algorithms 11, 327–337 (1996)

    MathSciNet  MATH  Google Scholar 

  251. Salam, A., Graves–Morris, P.R.: On the vector ε–algorithm for solving linear systems of equations. Numer. Algorithms 29, 229–247 (2002)

    MathSciNet  MATH  Google Scholar 

  252. Samuelson, P.A.: Interactions between the multiplier analysis and the principle of acceleration. Rev. Econ. Stat. 21, 75–78 (1939)

    Google Scholar 

  253. Samuelson, P.A.: A convergent iterative process. J. Math. and Phys. 24, 131–134 (1945)

    MathSciNet  MATH  Google Scholar 

  254. Schmidt, R.J.: On the numerical solution of linear simultaneous equations by an iterative method. Phil. Mag. 7, 369–383 (1941)

    MathSciNet  MATH  Google Scholar 

  255. Schneider, C.: Vereinfachte Rekursionen zur Richardson–Extrapolation in Spezialfällen. Numer. Math. 24, 177–184 (1975)

    MathSciNet  MATH  Google Scholar 

  256. Schur, I.: Über potenzreihen, die im Innern des Einheitskreises beschränkt sind. J. Reine Angew. Math. 147, 205–232 (1917)

    MathSciNet  MATH  Google Scholar 

  257. Senhadji, M.N.: On condition numbers of the Shanks transformation. J. Comput. Appl. Math. 135, 41–61 (2001)

    MathSciNet  MATH  Google Scholar 

  258. Shanks, D.: An Analogy between Transient and Mathematical Sequences and some Nonlinear Sequence–to–Sequence Transforms suggested by it. Part I, Memorandum 9994, Naval Ordnance Laboratory, White Oak (1949)

  259. Shanks, D.: Non–linear transformations of divergent and slowly convergent sequences. J. Math. and Phys. 34, 1–42 (1955)

    MathSciNet  MATH  Google Scholar 

  260. Shanks, D.: Solved and Unsolved Problems in Number Theory. Chelsea Publishing Company, New York (1962)

    MATH  Google Scholar 

  261. Sidi, A.: Convergence and stability properties of minimal polynomial and reduced rank extrapolation algorithms. SIAM J. Numer. Anal. 23, 197–209 (1986)

    MathSciNet  MATH  Google Scholar 

  262. Sidi, A.: Extrapolation vs. projection methods for linear systems of equations. J. Comp. Appl. Math. 22, 71–88 (1988)

    MathSciNet  MATH  Google Scholar 

  263. Sidi, A.: Application of vector extrapolation methods to consistent singular linear systems. Appl. Numer. Math. 6, 487–500 (1989/90)

  264. Sidi, A.: Efficient implementation of minimal polynomial and reduced rank extrapolation methods. J. Comp. Appl. Math. 36, 305–337 (1991)

    MathSciNet  MATH  Google Scholar 

  265. Sidi, A.: Practical Extrapolation Methods. Theory and Applications. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  266. Sidi, A.: Minimal polynomial and reduced rank extrapolation methods are related. Adv. Comput. Math. 43, 151–170 (2017)

    MathSciNet  MATH  Google Scholar 

  267. Sidi, A.: Vector Extrapolation Methods with Applications. SIAM, Philadelphia (2017)

    MATH  Google Scholar 

  268. Sidi, A., Bridger, J.: Convergence and stability analyses for some vector extrapolation methods in the presence of defective iteration matrices. J. Comput. Appl. Math. 22, 35–61 (1988)

    MathSciNet  MATH  Google Scholar 

  269. Sidi, A., Ford, W.F., Smith, D.A.: Acceleration of convergence of vector sequences. SIAM J. Numer. Anal. 23, 178–196 (1986)

    MathSciNet  MATH  Google Scholar 

  270. Skelboe, S.: Extrapolation Methods for Computation of the Periodic Steady–State Response of Nonlinear Circuits, Report IT 7, Institute of Circuit Theory and Telecommunication Technical University of Denmark (1976)

  271. Skelboe, S.: Extrapolation methods for computation of the periodic steady–state response of nonlinear circuits. In: Proc IEEE International Symposium on Circuits and Systems, pp. 64–67 (1977)

  272. Skelboe, S.: Computation of the periodic steady–state response of nonlinear networks by extrapolation methods. IEEE Trans. Circ. Syst. 27, 161–175 (1980)

    MathSciNet  MATH  Google Scholar 

  273. Smith, D.A., Ford, W.F., Sidi, A.: Extrapolation methods for vector sequences. SIAM Rev. 29, 199–233 (1987). Erratum, SIAM Rev. 30 623–624 (1988)

    MathSciNet  MATH  Google Scholar 

  274. Sonneveld, P.: A fast Lanczos–type solver for nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 10, 35–52 (1989)

    MathSciNet  MATH  Google Scholar 

  275. Steele, J.A.: Some Results concerning the Fundamental Nature of Wynn’s Vector Epsilon Algorithm, Ph.D. Thesis. University of Saskatchewan, Saskatoon (2002)

  276. Steele, J.A., Dolovich, A.T.: Toward the kernel of the vector epsilon algorithm. Int. J. Numer. Methods Engrg. 48, 721–730 (2000)

    MathSciNet  MATH  Google Scholar 

  277. Steffensen, J.F.: On certain inequalities between mean values, and their application to actuarial problems, Skand. Aktuarietids (Scand Actuar. J.) 1918:1, 82–97 (1918)

  278. Steffensen, J.F.: Interpolation. The Williams & Wilkins Company, Baltimore (1927)

    MATH  Google Scholar 

  279. Steffensen, J.F.: Remarks on iteration. Skand. Aktuarietidskr. 16, 64–72 (1933)

    MATH  Google Scholar 

  280. Sylvester, J.J.: On the relation between the minor determinants of linearly equivalent quadratic functions. Philos. Mag., s. IV, 1, 295–305, 415 (1851)

    Google Scholar 

  281. Szénássy, B.: History of Mathematics in Hungary until the 20th Century. Springer–Verlag, Berlin (1992)

    MATH  Google Scholar 

  282. Tan, R.C.E.: Implementation of the topological epsilon–algorithm. SIAM J. Sci. Stat. Comput. 9, 839–848 (1988)

    MATH  Google Scholar 

  283. Tempelmeier, U.: A new proof of the cross–rule for the ε–algorithm based on Schur complements. J. Comput. Appl. Math. 21, 55–61 (1988)

    MathSciNet  MATH  Google Scholar 

  284. Thiele, T.N.: Interpolationsrechnung. Teubner, Leipzig (1909)

    MATH  Google Scholar 

  285. Todd, J.: Survey of Numerical Analysis. McGraw–Hill, New York (1962)

    MATH  Google Scholar 

  286. Tornheim, L.: Convergence of multipoint iterative methods. J. Assoc. Comput. Mach. 11, 210–220 (1964)

    MathSciNet  MATH  Google Scholar 

  287. Tucker, R.R.: Error Analysis, Convergence, Divergence, and the Acceleration of Convergence. Ph.D. Thesis, Oregon State University, Corvallis (1963)

    Google Scholar 

  288. Tucker, R.R.: The δ 2–process and related topics. Pac. J. Math. 22, 349–359 (1967)

    MathSciNet  Google Scholar 

  289. Tucker, R.R.: A geometric derivation of Daniel Shanks e k transform, The Faculty Review. Bulletin of the Carolina A&T State University 65(3), 60–63 (Spring 1973)

  290. Tsujimoto, S., Nakamura, Y., Iwasaki, M.: The discrete Lotka–Volterra system computes singular values. Inverse Probl. 17, 53–58 (2001)

    MathSciNet  MATH  Google Scholar 

  291. Van Iseghem, J.: Vector Padé approximants. In: Vichnevetsky, R., Vignes, J. (eds.) Numerical Mathematics and Applications, pp. 73–77, North–Holland (1985)

  292. Van Iseghem, J.: An extended cross rule for vector Padé approximants. Appl. Numer. Math. 2, 143–155 (1986)

    MathSciNet  MATH  Google Scholar 

  293. Van Iseghem, J.: Vector orthogonal relations. Vector Q D–algorithm. J. Comput. Appl. Math. 19, 141–150 (1987)

    MathSciNet  MATH  Google Scholar 

  294. Van Iseghem, J.: Approximants de Padé Vectoriels, Thèse de Doctorat d’État ès Sciences Mathématiques, Université des Sciences et Techniques de Lille–Flandres–Artois (1987)

  295. Varadhan, R., Roland, Ch.: Squared Extrapolation Methods (SQUAREM): A New Class of Simple and Efficient Numerical Schemes for Accelerating the Convergence of the EM Algorithm, Johns Hopkins University, Department of Biostatistics Working Papers 11–19–2004 (2004)

  296. Vorobyev, Yu. V.: Method of Moments in Applied Mathematics. Gordon and Breach, New York (1965)

    Google Scholar 

  297. Walker, H.F., Ni, P.: Anderson acceleration for fixed–point iterations. SIAM J. Numer. Anal. 49, 1715–1735 (2011)

    MathSciNet  MATH  Google Scholar 

  298. Wegstein, J.H.: Accelerating convergence of iterative processes. Comm. ACM 1(6), 9–13 (1958)

    MATH  Google Scholar 

  299. Weiss, L., McDonough, R.N.: Prony’s method, z–transform, and Padé approximation. SIAM Rev. 5, 145–149 (1963)

    MathSciNet  MATH  Google Scholar 

  300. Weniger, E.J.: Weakly convergent expansions of a plane wave and their use in Fourier integrals. J. Math. Phys. 26, 276–291 (1985)

    MathSciNet  MATH  Google Scholar 

  301. Weniger, E.J.: Nonlinear sequence transformations for the acceleration of convergence and the summation of divergent series. Comput. Phys. Rep. 10, 189–371 (1989). and Los Alamos Preprint arXiv:math--ph/0306302

    Google Scholar 

  302. Williams, H.C.: Daniel Shanks (1917–1996). Not. Amer. Math. Soc. 44, 813–816 (1997)

    MathSciNet  MATH  Google Scholar 

  303. Wimp, J.: Derivative–Free Iteration Processes of Higher Order, Aerospace Research Laboratories, Office of Aerospace Research, United States Air Force, Vol. 69, Issue 183 (1969)

  304. Wimp, J.: Derivative–free iteration processes. SIAM J. Numer. Anal. 7, 329–334 (1970)

    MathSciNet  MATH  Google Scholar 

  305. Wimp, J.: Sequence Transformations and their Applications. Academic Press, New York (1981)

    MATH  Google Scholar 

  306. Wolfe, P.: The secant method for simultaneous non–linear equations. Comm. ACM 2, 12–13 (1959)

    MATH  Google Scholar 

  307. Wynn, P.: On a device for computing the e m(s n) transformation. Math. Tables Aids Comput. 10, 91–96 (1956)

    MathSciNet  MATH  Google Scholar 

  308. Wynn, P.: On a procrustean technique for the numerical transformation of slowly convergent sequences and series. Proc. Camb. Phil. Soc. 52, 663–671 (1956)

    MathSciNet  MATH  Google Scholar 

  309. Wynn, P.: On the propagation of error in certain non–linear algorithms. Numer. Math. 1, 142–149 (1959)

    MathSciNet  MATH  Google Scholar 

  310. Wynn, P.: Confluent forms of certain nonlinear algorithms. Arch. Math. (Basel) 11, 223–234 (1960)

    MATH  Google Scholar 

  311. Wynn, P.: A note on a confluent form of the ε–algorithm. Arch. Math. (Basel) 11, 237–240 (1960)

    MathSciNet  MATH  Google Scholar 

  312. Wynn, P.: Upon a second confluent form of the ε–algorithm. Proc. Glasgow Math. Assoc. 5, 160–165 (1961)

    MathSciNet  MATH  Google Scholar 

  313. Wynn, P.: Acceleration techniques for iterated vector and matrix problems. Math. Comp. 16, 301–322 (1962)

    MathSciNet  MATH  Google Scholar 

  314. Wynn, P.: On a connection between the first and the second confluent forms of the ε–algorithm. Niew. Arch. Wiskd. 11, 19–21 (1963)

    MathSciNet  MATH  Google Scholar 

  315. Wynn, P.: Singular rules for certain nonlinear algorithms. BIT 3, 175–195 (1963)

    MathSciNet  MATH  Google Scholar 

  316. Wynn, P.: Continued fractions whose coefficients obey a non–commutative law of multiplication. Arch. Ration. Mech. Anal. 12, 273–312 (1963)

    MATH  Google Scholar 

  317. Wynn, P.: Partial differential equations associated with certain non–linear algorithms. Z. Angew. Math. Phys. 15, 273–289 (1964)

    MathSciNet  MATH  Google Scholar 

  318. Wynn, P.: General purpose vector epsilon algorithm Algol procedures. Numer Math. 6, 22–36 (1964)

    MathSciNet  MATH  Google Scholar 

  319. Wynn, P.: An arsenal of Algol procedures for the evaluation of continued fractions and for effecting the epsilon algorithm. Chiffres 4, 327–362 (1966)

    MathSciNet  Google Scholar 

  320. Wynn, P.: Upon systems of recursions which obtain among the quotients of the Padé table. Numer. Math. 8, 264–269 (1966)

    MathSciNet  MATH  Google Scholar 

  321. Wynn, P.: On the convergence and stability of the epsilon algorithm. SIAM J. Numer. Anal. 3, 91–122 (1966)

    MathSciNet  MATH  Google Scholar 

  322. Wynn, P.: Upon a Conjecture Concerning a Method for Solving Linear Equations, and Certain Other Matters, MRC Technical Summary Report 626, Mathematics Research Center, United States Army, The University of Wisconsin, Madison, Wisconsin (1966)

  323. Wynn, P.: The Abstract Theory of the Epsilon Algorithm, Rapport CRM–74, Centre de Recherches Mathématiques, Université de Montréal (1971)

  324. Wynn, P.: Invariants associated with the epsilon algorithm and its first confluent form. Rend. Circ. Mat. Palermo, ser. II, 21, 31–41 (1972)

    MathSciNet  MATH  Google Scholar 

  325. Wynn, P.: Hierarchies of arrays and function sequences asociated with the epsilon algorithm and its first confluent form. Rend. Mat. Roma, ser. VI, 5(4), 819–852 (1972)

    MATH  Google Scholar 

  326. Wynn, P.: Upon some continuous prediction algorithms. I, Calcolo 9, 197–234; II, 235–278 (1973)

  327. Wynn, P.: Sur l’équation aux dérivées partielles de la surface de Padé. C.R. Acad. Sci. Paris, Sér. A 278, 847–850 (1974)

    MathSciNet  MATH  Google Scholar 

  328. Yamamoto, T.: Historical developments in convergence analysis for Newton’s and Newton–like methods. J. Comput. Appl. Math. 124, 1–23 (2000); also In: Brezinski, C., Wuytack, L. (eds.), Numerical Analysis: Historical Developments in the 20th Century, pp. 241–263. North–Holland, Amsterdam (2001)

  329. Zhang, F.-Z. (ed.): The Schur Complement and its Applications. Springer, New York (2005)

Download references

Acknowledgements

We are indebted to many colleagues for their help during the preparation of this paper. We would like to thank Stan Cabay, William Ford, Juan Antonio Pérez, Avram Sidi, David Smith, Yossi Stein, and Héctor René Vega–Carrillo for sharing with us their memories on their work, and allowing us to quote what they wrote us. We are also grateful to Matania Ben–Artzi, Michel Bercovier, Achiya Dax, Arieh Iserles, George Labahn and Stig Skelboe for their help in our search for information. Luc Vinet, Director of the Centre de Recherches Mathématiques of the Université de Montréal asked Vincent Masciotra, head of the Administration of this Center, to retrieve and sent us the administrative documents concerning Peter Wynn. We are grateful to them, and to Michel Delfour for his help. Thank you to Reuvena Shalhevet–Kaniel for sending us a photo of her husband. Peter Pulay was kind enough to allow us to reproduce what he wrote us on his work. We would like to thank Donald G.M. Anderson for sending us a long commentary about the discovery of his acceleration method, and, moreover, to carefully reading our paper and suggesting many important improvements. Susan Virginia Welby was quite helpful during our exchanges with Donald G.M. Anderson. We thank her. We appreciated the help of Gérard Meurant who carefully checked the paper, and whose remarks greatly helped us to improve it. Hervé Le Ferrand also suggested many useful modifications. Our paper substantially benefited from the relevant remarks of Ernst Joachim Weniger. We are grateful to him. Jean–Claude Miellou reminded us some interesting references. With Michel Crouzeix we looked again at our common paper of 1970, and we produced a better proof of one of our results. We thank him. Finally, we are indebted to Stefano Cipolla who was able to find on the internet all the references we needed. Without his help, it would have been much more difficult to complete this paper.

Funding

The work of C.B. was supported by the Labex CEMPI (ANR-11-LABX-0007-01). M.R.–Z. is a member of the INdAM Research group GNCS, which partially supported the research.

Author information

Authors and Affiliations

  1. Laboratoire Paul Painlevé, UMR CNRS 8524, UFR de Mathématiques, Université de Lille, 59655, Villeneuve d’Ascq cedex, France

    Claude Brezinski

  2. Dipartimento di Matematica “Tullio Levi-Civita”, Università degli Studi di Padova, Via Trieste 63, 35121, Padova, Italy

    Michela Redivo–Zaglia

Authors
  1. Claude Brezinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michela Redivo–Zaglia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claude Brezinski.

Additional information

We dedicate this paper to the memory of Peter Wynn (1931–2017)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brezinski, C., Redivo–Zaglia, M. The genesis and early developments of Aitken’s process, Shanks’ transformation, the ε–algorithm, and related fixed point methods. Numer Algor 80, 11–133 (2019). https://doi.org/10.1007/s11075-018-0567-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-018-0567-2

Keywords

Mathematics Subject Classification (2010)

Search

Navigation