Skip to main content
SpringerLink
Log in

Another Hager-Zhang-type method via singular-value study for constrained monotone equations with application

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Without setting any condition on the parameter \(\theta _k\) of a four-term version of the classical one-parameter Hager-Zhang (HZ) method, this article proposes another HZ-type scheme for solving constrained monotone equations, where the condition for global convergence is satisfied for \(\theta _k\in [0,+\infty )\). This is an improvement from the former, its recent adaptive variant, where the global convergence condition holds for \(\theta _k\in (0,+\infty )\) under certain defined condition, as well as other adaptations for systems of monotone equations, where the condition holds only when \(\theta _k\in (\frac{1}{4},+\infty )\). By conducting singular value study of iteration matrix of the scheme, a choice of \(\theta _k\) restricted in the interval \((0,\frac{1}{4}]\) is obtained to study its impact on the scheme. Moreover, the scheme converges globally and its effectiveness is shown by some numerical experiments and image de-blurring application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of Supporting Data

All data related to work are included in the manuscript.

References

  1. Yuan, G., Lu, J., Wang, Z.: The PRP conjugate gradient algorithm with a modified WWP line search and its application in the image restoration problems. Appl. Numer. Math. 152, 1–11 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  2. Yuan, G., Lu, J., Wang, Z.: The modified PRP conjugate gradient algorithm under a non-descent line search and its application in the Muskingum model and image restoration problems. J. Soft Comput. 25, 5867–5879 (2021)

    Article  MATH  Google Scholar 

  3. Yuan, G., Li, T., Hu, W.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  4. Aminifard, Z., Babaie-Kafaki, S.: Dai-Liao extensions of a descent hybrid nonlinear conjugate gradient method with application in signal processing. Numer. Alg. 89, 1369–1387 (2022). https://doi.org/10.1007/s11075-021-01157-y

  5. Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 35–58 (2006)

    MathSciNet  MATH  Google Scholar 

  6. Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Research Nat. Bur. Standards. 49, 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  8. Barzilai, J., Borwein, J.M.: Two point step size gradient method. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hager, W.W., Zhang, H.: Algorithm 851: \(CG_{-}{Descent}\), a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32(1), 113–137 (2006)

    Article  MATH  Google Scholar 

  11. Nocedal, J.: Updating quasi-Newton matrixes with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MATH  Google Scholar 

  12. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Programming Ser. B 45, 503–528 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2, 21–42 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  14. Liu, J.K., Li, S.J.: A projection method for convex constrained monotone nonlinear equations with applications. Comput. Math. Appl. 70(10), 2442–2453 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Xiao, Y., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405(1), 310–319 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  16. Waziri, M.Y., Ahmed, K., Halilu, A.S., Sabi’u, J.: Two new Hager-Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing. Rairo Oper Res. 56, 239–273 (2022). https://doi.org/10.1051/ro/2021190

  17. Waziri, M.Y., Ahmed, K., Halilu, A.S., Awwal, A.M.: Modified Dai-Yuan iterative scheme for nonlinear systems and its application. Numer. Algor. Control Optim. 13(1), 53–80 (2023). https://doi.org/10.3934/naco.2021044

  18. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A family of Hager-Zhang conjugate gradient methods for system of monotone nonlinear equations. Appl. Math. Comput. 361, 645–660 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  19. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. 9, 443–457 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  20. Waziri, M.Y., Ahmed, K., Sabi’u, J.: Descent Perry conjugate gradient methods for systems of monotone nonlinear equations. Numer. Algor. 85, 763–785 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  21. Waziri, M.Y., Ahmed, K., Sabi’u, J., Halilu, A.S.: Enhanced Dai-Liao conjugate gradient methods for systems of monotone nonlinear equations. SeMA J. 78, 15–51 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  22. Sabi’u, J., Shah, A., Waziri, M.Y.: Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations. Appl. Numer. Math. 153, 217–233 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  23. Sabi’u, J., Shah, A., Waziri, M.Y., Ahmed, K.: Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint. Int. J. Comput Methods. (2020). https://doi.org/10.1142/S0219876220500437

    Article  MATH  Google Scholar 

  24. Sabi’u, J., Shah, A., Waziri, M.Y.: A modified Hager-Zhang conjugate gradient method with optimal choices for solving monotone nonlinear equations. Int. J. Comput Math. (2021). https://doi.org/10.1080/00207160.2021.1910814

    Article  MATH  Google Scholar 

  25. Waziri, M.Y., Usman, H., Halilu, A.S., Ahmed, K.: Modified matrix-free methods for solving systems of nonlinear equations. Optimization. (2020). https://doi.org/10.1080/02331934.2020.1778689

    Article  MATH  Google Scholar 

  26. Waziri, M.Y., Ahmed, K: Two descent Dai-Yuan conjugate gradient methods for systems of monotone nonlinear equations. J. Sci. Comput. 90(36), (2022). https://doi.org/10.1007/s10915-021-01713-7

  27. Halilu, A.S., Majumder, A., Waziri, M.Y., Awwal, A.M., Ahmed, K.: On solving double direction methods for convex constrained monotone nonlinear equations with image restoration. Comput. Appl. Math. 40 239, (2021)

  28. Halilu, A.S., Majumder, A., Waziri, M.Y., Ahmed, K., Awwal, A.M.: Motion control of the two joint planar robotic manipulators through accelerated Dai-Liao method for solving system of nonlinear equations. Eng. Comput. 39(5), 1802–1840 (2022). https://doi.org/10.1108/EC-06-2021-0317

  29. Ahmed, K., Waziri, M.Y., Halilu, A.S., Murtala, S., Sabi’u, J.: On a scaled symmetric Dai-Liao-type scheme for constrained system of nonlinear equations with applications. J. Optim. Theory Appl. (2023). https://doi.org/10.1007/s10957-023-02281-6

    Article  Google Scholar 

  30. Ahmed, K., Waziri, M.Y., Halilu, A.S., Murtala, S.: Sparse signal reconstruction via Hager-Zhang-type schemes for constrained system of nonlinear equations. Optimization (2023). https://doi.org/10.1080/02331934.2023.2187255

    Article  Google Scholar 

  31. Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L. (eds.) Reformulation: Nonsmooth, pp. 355–369. Kluwer Academic Publishers, Piecewise Smooth, Semismooth and Smoothing Methods (1998)

    Google Scholar 

  32. Zhang, L., Zhou, W.: Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196, 478–484 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhou, W.J., Li, D.H.: Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25, 89–96 (2007)

    MathSciNet  Google Scholar 

  34. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comp. 35, 773–782 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  35. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23, 296–320 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wang, C., Wang, Y., Xu, C.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Methods Oper. Res. 66, 33–46 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  37. Andrei, N.: Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34(2), 319–330 (2011)

    MathSciNet  MATH  Google Scholar 

  38. Babaii-Kafaki, S., Ghanbari, R.: A class of descent four-term extension of the Dai-Liao conjugate gradient method based on the scaled memoryless BFGS update. J. Indust. Manag. Optim. 13(2), 649–658 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  39. Zhang, L., Zhou, W., Li, D.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22, 697–711 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  40. Alhawarat, A., Alhamzi, G., Masmali, I., Salleh, Z.: A descent four-term conjugate gradient method with global convergence properties for large-scale unconstrained optimisation problems. Math. Prob. Eng. Article ID 6219062, 1–14 (2021). https://doi.org/10.1155/2021/6219062

    Article  MATH  Google Scholar 

  41. Alhawarat, A., Alolaiyan, H., Ibtisam, A., Salleh, Z., Ismail, S.: A descent four-term of Liu and Storey conjugate gradient method for large scale unconstrained optimization problems. Euro. J. Pure Appl. Math. 14(4), 1429–1456 (2021)

    Article  MathSciNet  Google Scholar 

  42. Hawraz, N.J., Khalil, K., Hisham, M.A.: Four-term conjugate gradient method based on pure conjugacy condition for unconstrained optimization. Kirkuk university journal and scientific studies. 13(2), 101–113 (2018)

    Article  Google Scholar 

  43. Zhou, W.J., Li, D.H.: Limited memory BFGS methods for nonlinear monotone equations. J. Comput. Math. 25, 89–96 (2007)

    MathSciNet  Google Scholar 

  44. Babai-Kafaki, S.: On the sufficient descent condition of the Hager-Zhang conjugate gradient methods. 40R-Q. J. Oper. Res. 12, pp. 285–292 (2014)

  45. Babai-Kafaki, S., Ghanbari, R.: An adaptive Hager-Zhang conjugate gradient method. Filomat 30(14), 3715–3723 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  46. Goncalves, M.L.N., Prudente, L.F.: On the extension of the Hager-Zhang conjugate gradient method for vector optimization, technical report, (2018)

  47. Ahmed, I.H., Al-naemi, G.M.: A descent modified Hager-Zhang conjugate gradient method and its global convergence. Iraqi J. Statistical Sci. 20, 222–236 (2011)

    Google Scholar 

  48. Liu, H., Wang, H., Ni, Q.: On Hager and Zhang’s conjugate gradient method with guaranteed descent. Appl. Math. Comput. 236, 400–407 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  49. La Cruz, W., Raydan, M.: Nonmonotone spectral methods for large-scale nonlinear systems. Optim. Methods Softw. 180(5), 583–599 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  50. Wang, C.W., Wang, Y.J., Xu, C.L.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Meth. Oper. Res. 66, 33–46 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  51. Yin, J., Jian, J., Jiang, X., Liu, M., Wang, L.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Alg. 88, 389–418 (2021). https://doi.org/10.1007/s11075-020-01043-z

  52. Hu, Y., Wang, Y.: An efficient projected gradient method for convex constrained monotone equations with applications in compressive sensing. J. Appl. Math. Physics. 8, 983–998 (2020)

    Article  Google Scholar 

  53. Koorapetse, M., Kaelo, P., Lekoko, S., Diphofu, T.: A derivative-free RMIL conjugate gradient projection method for convex constrained nonlinear monotone equations with applications in compressive sensing. Appl. Numer. Math. 165, 431–441 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  54. Yuan, N.: A derivative-free projection method for solving convex constrained monotone equations. ScienceAsia. 43, 195–200 (2017)

    Article  Google Scholar 

  55. Gao, P., He, C.: An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints. Calcolo 55, 53 (2018). https://doi.org/10.1007/s10092-018-0291-2

    Article  MathSciNet  MATH  Google Scholar 

  56. Abubakar, A.B., Kumam, P., Mohammed, H., Sitthithakerngkiet, K.: A modified Fletcher-Reeves conjugate gradient method for monotone nonlinear equations with some applications. Mathematics. 7(745), 1–25 (2019)

    Google Scholar 

  57. Sun, W., Yuan, Y.X.: Optimization theory and methods: nonlinear programming. Springer, New York (2006)

    MATH  Google Scholar 

  58. Aminifard, Z., Hosseini, A., Babaie-Kafaki, S.: Modified conjugate gradient method for solving sparse recovery problem with nonconvex penalty. Signal Processing 193, 108424 (2022)

    Article  Google Scholar 

  59. Peiting, G., Chuanjiang, H.: A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations. Optimization. J. Math. Prog. Oper. Res. 1–18 (2018)

  60. Raymond, H.C., Chung-Wa, H., Nikolova, M.: Salt-and-pepper noise removal by median type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)

    Article  Google Scholar 

  61. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2, Ser. A), 20–2013 (2002)

  62. La Cruz, W.: A Spectral algorithm for large-scale systems of nonlinear monotone equations. Numer. Algor. (2017). https://doi.org/10.1007/s1107s-017-0299-8

  63. Figueiredo, M., Nowak, R., Wright, S.J.: Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems, pp. 586–597. IEEE J-STSP IEEE Press, Piscataway, NJ (2007)

    Google Scholar 

  64. Pang, J.S.: Inexact Newton methods for the nonlinear complementarity problem. Math. Program. 1, 54–71 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  65. Yuan, G., Yang, H., Zhang, M.: Adaptive three-term PRP algorithms without gradient Lipschitz continuity condition for nonconvex functions. Numer. Algor. 91(1), 145–160 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  66. Yuan, G., Li, P., Lu, J.: The global convergence of the BFGS method with a modified WWP line search for nonconvex functions. Numer. Algor. 91(1), 353–365 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  67. Yuan, G., Zhang., M, Zhou, Y.: Adaptive scaling damped BFGS method without gradient Lipschitz continuity. Appl. Math. Letters. 124, 107634 (2022)

  68. Xiao, Y., Wang, Q., Hu, Q.: Non-smooth equations based method for l1-norm problems with applications to compressed sensing. Nonlinear Anal. Theory Methods Appl. 74(11), 3570–3577 (2011)

    Article  MATH  Google Scholar 

  69. Watkin, D.S.: Fundamentals of matrix computations. Wiley, New York (2002)

    Book  Google Scholar 

  70. Alfred, M.B., David, L.D., Michael, E.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review. 51(1), 34–81 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank members of the Numerical optimization research group, Bayero university, Kano for their advise and support.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Bayero University, Kano, Nigeria

    Kabiru Ahmed & Mohammed Yusuf Waziri

  2. Department of Mathematics, Sule Lamido University, Kafin Hausa, Nigeria

    Abubakar Sani Halilu

  3. Department of Mathematics, Federal University, Dutse, Nigeria

    Salisu Murtala

  4. Department of Mathematics, Yusuf Maitama Sule University, Kano, Nigeria

    Jamilu Sabi’u

  5. Numerical Optimization Research Group, Bayero University, Kano, Nigeria

    Kabiru Ahmed, Mohammed Yusuf Waziri, Abubakar Sani Halilu, Salisu Murtala & Jamilu Sabi’u

Authors
  1. Kabiru Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Yusuf Waziri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abubakar Sani Halilu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salisu Murtala
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jamilu Sabi’u
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Kabiru Ahmed: Conceptualization, Methodology, Validating, Software, Data curation, Writing - original draft, Writing - review & editing. Mohammed Yusuf Waziri: Formal analysis, Investigation, Writing - review & editing, Supervision. Abubakar Sani Halilu: Writing - review & editing, Supervision. Salisu Murtala: Writing - review & editing, Supervision. Jamilu Sabi’u: Editing & Supervision.

Corresponding author

Correspondence to Kabiru Ahmed.

Ethics declarations

Ethical Approval

Not applicable.

Competing interest

The authors declare that there is no competing financial and/ or non financial interests in relation to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, K., Waziri, M.Y., Halilu, A.S. et al. Another Hager-Zhang-type method via singular-value study for constrained monotone equations with application. Numer Algor (2023). https://doi.org/10.1007/s11075-023-01678-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11075-023-01678-8

Keywords

Mathematics Subject Classification (2010)

Search

Navigation