Abstract

For systems of nonlinear equations, a modified efficient Levenberg–Marquardt method with new LM parameters was developed by Amini et al. (2018). The convergence of the method was proved under the local error bound condition. In order to enhance this method, using nonmonotone technique, we propose a new Levenberg–Marquardt parameter in this paper. The convergence of the new Levenberg–Marquardt method is shown to be at least superlinear, and numerical experiments show that the new Levenberg–Marquardt algorithm can solve systems of nonlinear equations effectively.

1. Introduction

Consider the system of nonlinear equationswhere the function is continuously differentiable. In this paper, we assume that the solution set of (1) (denoted by ) is nonempty, with referring to the 2-norm.

Newton method is an important method to solve system (1) in [1]. At each iteration, it uses the trial stepwhere and is Jacobian matrix. When is Lipschitz continuous and nonsingular, then the convergence of this method is quadratic at the solution. However, trial step may not exist and is singular or near singular. Newton method may not be well defined. To overcome this difficulty, Levenberg–Marquardt (LM) method was created by Levenberg [2] and Marquardt [3] which uses the trial step at each iteration, whereand is a nonnegative constant. By introducing a nonnegative parameter , LM method overcomes the problem that is singular or near singular; furthermore, excessive step size is avoided. In this case, where and Jacobian matrix is nonsingular, the LM method is reduced to Newton method.

The efficiency of the LM method is affected by the parameter . For example, let , under the local error bound condition, the LM method is shown to have quadratic convergence by Yamashita and Fukushima in [4]. However, when the sequence is far away from the set , may be very big which may lead to large . It will result in a smaller LM step size, further reducing the efficiency of the algorithm. In [5], Fan used , where is updated with trust region technology in each iteration, the LM method also has quadratic convergence under some suitable conditions, and can alleviate the effect of the initial point being far away from the set .

To avoid this trouble, Amini used in [6]; when the sequence is far from the solution set and is very large, is close to , which effectively controls the range of . Umar proposed some new LM parameters , in [7]. Wang used with updated by trust region techniques from iteration to iteration in [8].

Ma introduced into LM method and used a new LM parameter in [9], where . It is noticeable that is a convex combination of and , and the quadratic convergence of this method is proved. There are numerous other various LM methods to solve (1); interested readers are referred to [10–12] for related work. In order to discuss the range of parameter , inspired by [6, 8, 9], in this paper, we choose a new LM parameter as follows:where is a convex combination of and and is updated with trust region technology in each iteration.

Now, we setas the merit function for (1). We define the actual reduction and the predicted reduction of at the th iteration as follows:where is computed by (3). The following ratio is

Grippo applied the nonmonotone line search technique to Newton’s method in [13]; some authors have extended the nonmonotone techniques to trust region algorithm and proposed a lot of effective nonmonotone trust region methods in [14, 15]. And Amini proposed nonmonotone line search technique for the LM method in [6]. Numerical experiments show that the algorithm with the nonmonotone technique is more efficient than the algorithm without the nonmonotone technique. Inspired by these theories, we apply a nonmonotone strategy to LM method in this paper. Let us replace actual reduction (6) with the following actual reduction:where, and is a positive integer constant. Obviously, by this change, will be compared with the in each iteration, further leading to affect the ratio. The ratio after the change is

It can be used to decide whether the trial step is accepted and update the trust region parameter .

The paper is organized as follows. In Section 2, we present a new algorithm and then prove the global convergence of the new algorithm under some conditions. In Section 3, under the local error bound condition, the convergence of the new Levenberg–Marquardt method is shown to be at least superlinear. In Section 4, the new algorithm is an effective algorithm, which is demonstrated by numerical results. At last, we give some conclusions in Section 5.

2. The Efficient Algorithm and Global Convergence

In this section, firstly, we present the new efficient LM algorithm and then prove the global convergence of the new algorithm.

When sequence is close to the solution, the steps may be too large, so we requirein the new algorithm, where is a positive constant, and this is implemented by Step 5.

Lemma 1. For all , we have

Proof. This proof is directly derived from the important theory given by Powell in [16].
From literature [6], the following lemma can be obtained.

Lemma 2. Suppose the sequence be generated by Algorithm 1, then the sequence converges.

Next we present some of the assumptions needed in the following content.

Assumption 1. (a) is continuously differentiable and Lipschitz continuous; i.e., there exists a positive constant that makes(b) is Lipschitz continuous; i.e., there exists a positive constant such that

Lemma 3. If Assumption 1 holds, then we have

Proof. The proof of (17) can be found in [17]. So, we only prove (16). Using mean value theorem, there exists that makesand hence,According to the last equation, we can obtainSo (16) is true.

Theorem 1. Suppose that Assumption 1 is true. Then, Algorithm 1 terminates in finite iterations or satisfies

Proof. Assume that the theorem is incorrect; then, there exist a positive constant and a constant that makesFirstly, we prove thatSince is accepted by the algorithm, we haveThis, along with (17), (22), and Lemma 1, for all , that meansReplacing with , for all sufficiently large , there isFrom Lemma 2which together with the last inequality yields is a positive constant, soUsing Assumption 1, the last equality implies thatLet . Using induction, for all , we can show thatFor , we can have from (31) that (29) is true. Assuming that (29) is true for given , we show that (29) holds for given . Let be large enough such that . Substituting with and using (24), we obtainSimilarly, we can deduce thatTherefore (31) holds. Along with Assumption 1 we imply thatSimilarly, for any given , we have . On the one hand, for any , we haveUsing (31) and the fact that , we haveWith Assumption 1, we concludeAnd then (22) is proved. By using (22) and (25), we can deduce thatThen, it follows from (3) in Algorithm 1, (17), and (22) thatOn the other hand, by (6), (8), (17), (22), (38), and Lemma 1, we can deduce thatAnd then combined with (8) and (10), we can see thatConsidering Algorithm 1, obviously, for all large , there exists a positive constant such that , which conflicts with (22) and so Theorem 1 is true.

3. Local Convergence

Definition 1. Let be a subset of that makes . We say provides a local error bounded on for (1), where , if there exists a positive constant such that

Assumption 2. (a) is continuously differentiable, and provides a local error bound on subset for problem (1), where(b)andare both Lipschitz continuous on; that is, there exist two positive constants, that makewhich implies

Lemma 4. Suppose that Assumption 2 is true. Then, for all sufficiently large , we have the following.(1)There exists a positive constant that makes(2), where .

Proof. The proof process of (1) is the same as that of Lemma 3.2 in [6], so we are not going to prove it here and only give the proof of (2).where , obviously,We can obtain from Definition 1 and (45) and (46) thatThus we obtainThen we show the following inequalities:If , then the following holds:and if , thenThrough the above two inequalities, we haveSimilarly, if , then the following holdsand if , we haveHence, we obtainFrom Algorithm 1, (52), and (58), we havewhere . So when is sufficiently small,