Convergence of the Exponentiated Gradient Method with Armijo Line Search

Abstract

Consider the problem of minimizing a convex differentiable function on the probability simplex, spectrahedron, or set of quantum density matrices. We prove that the exponentiated gradient method with Armijo line search always converges to the optimum, if the sequence of the iterates possesses a strictly positive limit point (element-wise for the vector case, and with respect to the Löwner partial ordering for the matrix case). To the best of our knowledge, this is the first convergence result for a mirror descent-type method that only requires differentiability. The proof exploits self-concordant likeness of the log-partition function, which is of independent interest.

Appendices

Appendix A Inapplicability of Existing Convergence Guarantees to Quantum State Tomography

Quantum state tomography is the task of estimating the state of a quantum systems, which is essential to calibrating quantum computation devices [4, 30]. Numerically, it corresponds to solving (P) with the objective function

$$\begin{aligned} f_{\text {QST}} ( \rho ) := - \sum _{i = 1}^n \log {{\mathrm{\mathrm {Tr}}}}( M_i \rho ) , \end{aligned}$$

where $M_i$ are positive semi-definite matrices given by the experimental data.

The following proposition shows that existing convergence guarantees for the EG method do not apply to quantum state tomography.

Proposition A.1

The function $f_{\text {QST}}$ is not Lipschitz, its gradient is not Lipschitz, and it is not smooth relative to the negative von Neumann entropy.

Proof

Consider the two-dimensional case, where $\rho = ( \rho _{i, j} )_{1 \le i, j \le 2} \in {\mathbb {C}}^{2 \times 2}$. Define $e_1 := ( 1, 0 )$ and $e_2 := ( 0, 1 )$. Suppose that there are only two summands, with $M_1 = e_1 \otimes e_1$ and $M_2 = e_2 \otimes e_2$. Then, we have $f ( \rho ) = - \log \rho _{1,1} - \log \rho _{2,2}$. It suffices to disprove all properties for this specific f on the set of diagonal density matrices. Hence, we will focus on the function $g ( x, y ) := - \log x - \log y$, defined for any $x, y > 0$ such that $x + y = 1$.

As either x or y can be arbitrarily close to zero, g cannot be Lipschitz continuous in itself or its gradient due to the logarithmic functions. Define the entropy function

$$\begin{aligned} h(x, y) := - x \log x - y \log y + x + y, \end{aligned}$$

with the convention $0 \log 0 = 0$. Then, g is L-smooth relative to the relative entropy, if and only if $- L h - g$ is convex. It suffices to check the positive semi-definiteness of the Hessian of $- L h - g$. A necessary condition for the Hessian to be positive semi-definite is that

$$\begin{aligned} - L \frac{\partial ^2 h}{\partial x^2} ( x, y ) - \frac{\partial ^2 g}{\partial x^2} ( x, y ) = \frac{L}{x} - \frac{1}{x^2} \ge 0 , \end{aligned}$$

for all $x \in ] 0, 1 [$, which cannot hold for $x < ( 1 / L )$, for any fixed $L > 0$. $\square$

We note that similar objective functions can be found in positive linear inverse problems, positron emission tomography, portfolio selection, and Poisson phase retrieval [31,32,33,34].

Appendix B Technical Lemmas Necessary for Sect. 3

Define

$$\begin{aligned} \rho ( \alpha ) := C_\rho ^{-1} \exp \left[ \log ( \rho ) - \alpha \nabla f ( \rho ) \right] , \end{aligned}$$

for every non-singular $\rho \in {\mathcal {D}}$ and $\alpha \ge 0$, where $C_\rho$ is the positive real number normalizing the trace of $\rho ( \alpha )$.

Lemma B.1

For every non-singular $\rho \in {\mathcal {D}}$ and $\alpha > 0$, it holds that

$$\begin{aligned} {\langle { \nabla f ( \rho ), \rho ( \alpha ) - \rho }\rangle } \le - \frac{H ( \rho ( \alpha ), \rho )}{ \alpha } . \end{aligned}$$

Proof

The equivalent formulation of the EG method, (2), implies that

$$\begin{aligned} \alpha {\langle { \nabla f ( \rho ), \rho ( \alpha ) - \rho }\rangle } + H ( \rho ( \alpha ), \rho ) \le \alpha {\langle { \nabla f ( \rho ), \rho - \rho }\rangle } + H ( \rho , \rho ) = 0 . \end{aligned}$$

$\square$

Lemma B.2

Let $\rho \in {\mathcal {D}}$ be non-singular. If $\rho$ is a minimizer of f on ${\mathcal {D}}$, then $\rho ( \alpha ) = \rho$ for all $\alpha \ge 0$. If $\rho ( \alpha ) = \rho$ for some $\alpha > 0$, then $\rho$ is a minimizer of f on ${\mathcal {D}}$.

Proof

The optimality condition says that $\rho \in {{\mathrm{\mathrm {int}}}}{\mathcal {D}}$ is a minimizer of f on ${\mathcal {D}}$, if and only if

$$\begin{aligned} {\langle { \nabla f ( \rho ), \sigma - \rho }\rangle } \ge 0 , \quad \forall \sigma \in {\mathcal {D}} . \end{aligned}$$

For any $\alpha > 0$, we can equivalently write

$$\begin{aligned} {\langle { \alpha \nabla f ( \rho ) + \left[ \nabla h ( \rho ) - \nabla h ( \rho ) \right] , \sigma - \rho }\rangle } \ge 0 , \quad \forall \sigma \in {\mathcal {D}} , \end{aligned}$$

(9)

where h denotes the negative von Neumann entropy function, i.e.,

$$\begin{aligned} h ( \rho ) := {{\mathrm{\mathrm {Tr}}}}( \rho \log \rho ) - {{\mathrm{\mathrm {Tr}}}}\rho . \end{aligned}$$

Note that the quantum relative entropy H is the Bregman divergence induced by the negative von Neumann entropy. It is easily checked, again by the optimality condition, that (9) is equivalent to

$\square$

For every non-singular $\rho \in {\mathcal {D}}$ and $\alpha \ge 0$, define

$$\begin{aligned} G := - \nabla f ( \rho ), \quad H_\alpha := \log \rho + \alpha G. \end{aligned}$$

Let $G = \sum _j \lambda _j P_j$ be the spectral decomposition of G. Define $\eta _\alpha$ as a random variable satisfying

$$\begin{aligned} {\mathsf {P}} \left( \eta _\alpha = \lambda _j \right) = \frac{ {{\mathrm{\mathrm {Tr}}}}\left( P_j \exp ( H_\alpha ) \right) }{ {{\mathrm{\mathrm {Tr}}}}\exp ( H_\alpha ) }; \end{aligned}$$

(10)

it is easily checked that ${\mathsf {P}} \left( \eta _\alpha = \lambda _j \right) > 0$ for all j, and the probabilities sum to one.

Lemma B.3

For any $\alpha \in {\mathbb {R}}$, it holds that

$$\begin{aligned} \varphi ' ( \alpha ) = {\mathsf {E}}\, \eta _\alpha , \quad \varphi '' ( \alpha ) = {\mathsf {E}} \left( \eta _\alpha - {\mathsf {E}}\, \eta _\alpha \right) ^2, \quad \varphi ''' ( \alpha ) = {\mathsf {E}} \left( \eta _\alpha - {\mathsf {E}}\, \eta _\alpha \right) ^3 . \end{aligned}$$

Proof

Note that

$$\begin{aligned} {\mathsf {E}}\, \eta _\alpha ^n = \frac{{{\mathrm{\mathrm {Tr}}}}( G^n \exp ( H_\alpha ) )}{{{\mathrm{\mathrm {Tr}}}}\exp ( H_\alpha )} , \end{aligned}$$

for any $n \in {\mathbb {N}}$. Define $\sigma _\alpha := \exp ( H_\alpha ) / {{\mathrm{\mathrm {Tr}}}}\exp ( H_\alpha )$. A direct calculation gives

$$\begin{aligned} \varphi ' ( \alpha )&= {{\mathrm{\mathrm {Tr}}}}( G \sigma _\alpha ) , \quad \varphi '' ( \alpha ) = {{\mathrm{\mathrm {Tr}}}}( G^2 \sigma _\alpha ) - \left( {{\mathrm{\mathrm {Tr}}}}( G \sigma _\alpha ) \right) ^2 , \\ \varphi ''' ( \alpha )&= {{\mathrm{\mathrm {Tr}}}}( G^3 \sigma _\alpha ) - 3 {{\mathrm{\mathrm {Tr}}}}( G^2 \sigma _\alpha ) {{\mathrm{\mathrm {Tr}}}}( G \sigma _\alpha ) + 2 \left( {{\mathrm{\mathrm {Tr}}}}( G \sigma _\alpha ) \right) ^3 . \end{aligned}$$

The lemma follows. $\square$

Since $\eta _\alpha$ is a bounded random variable, it follows that $\varphi ''$ is bounded from above.

Corollary B.1

It holds that $\varphi '' ( \alpha ) \le ( 1 / 4 ) \varDelta ^2$, where

$$\begin{aligned} \varDelta := \lambda _{\max } ( \nabla f ( \rho ) ) - \lambda _{\min } ( \nabla f ( \rho ) ) . \end{aligned}$$

Proof

Recall that the variance of a random variable taking values in [a, b] is bounded from above by $( b - a )^2 / 4$.

$\square$

Appendix C Proof of Lemma 3.3

Recall the random variable $\eta _\alpha$ defined in (10). Suppose that $\varphi '' ( \alpha ) = 0$ for some $\alpha \in [ 0, + \infty [$. Then, we have $\eta _\alpha = 0$ almost surely, but this implies that $\varDelta = 0$, a contradiction. Therefore, we have $\varphi '' ( \alpha ) > 0$ for all $\alpha \in [ 0, + \infty [$.

We prove a general result. Let $\psi : {\mathbb {R}} \rightarrow {\mathbb {R}}$ be a $\mu$-self-concordant-like function. Suppose that $\psi '' ( t ) > 0$ for all t. Consider the function

$$\begin{aligned} \chi ( t ) := \log \left( \psi '' ( t ) \right) . \end{aligned}$$

We write, by the self-concordant likeness of $\psi$, that

$$\begin{aligned} \vert \chi ' ( t ) \vert = \frac{\vert \psi ''' ( t ) \vert }{ \psi '' ( t ) } \le \mu \, , \quad \forall t \in {\mathbb {R}} \, . \end{aligned}$$

Then, for any $t_1, t_2 \in {\mathbb {R}}$, we have

$$\begin{aligned} \vert \chi ( t_1 ) - \chi ( t_2 ) \vert = \left| \log \left( \psi '' ( t_1 ) \right) - \log \left( \psi '' ( t_2 ) \right) \right| \le \mu \vert t_2 - t_1 \vert \, ; \end{aligned}$$

that is,

$$\begin{aligned} \mathrm {e}^{- \mu \vert t_2 - t_1 \vert } \psi '' ( t_2 ) \le \psi '' ( t_1 ) \le \mathrm {e}^{ \mu \vert t_2 - t_1 \vert } \psi '' ( t_2 ) \, . \end{aligned}$$

Applying the Newton–Leibniz formula, we obtain

$$\begin{aligned} \psi ' ( t_2 ) - \psi ' ( t_1 )&= \int _{0}^1 \psi '' ( t_1 + \tau ( t_2 - t_1 ) ) ( t_2 - t_1 ) \, \mathrm {d}\tau \\&\le \int _0^1 \mathrm {e}^{\mu \tau \vert t_2 - t_1 \vert } \psi '' ( t_1 ) ( t_2 - t_1 ) \, \mathrm {d}\tau \\&= \left( \frac{\mathrm {e}^{\mu \vert t_2 - t_1 \vert } - 1}{ \mu \vert t_2 - t_1 \vert } \right) \psi '' ( t_1 ) ( t_2 - t_1 ) \, ; \end{aligned}$$

similarly, we obtain

$$\begin{aligned} \psi ' ( t_2 ) - \psi ' ( t_1 ) \ge - \left( \frac{\mathrm {e}^{- \mu \vert t_2 - t_1 \vert } - 1}{ \mu \vert t_2 - t_1 \vert } \right) \psi '' ( t_1 ) ( t_2 - t_1 ) \, . \end{aligned}$$

Applying the Newton–Leibniz formula again, we obtain

$$\begin{aligned} \psi ( t_2 ) - \psi ( t_1 )&= \int _0^1 \psi ' ( t_1 + \tau ( t_2 - t_1 ) ) ( t_2 - t_1 ) \, \mathrm {d}\tau \\&= \psi ' ( t_1 ) ( t_2 - t_1 ) + \int _0^1 \left( \psi ' ( t_1 + \tau ( t_2 - t_1 ) ) - \psi ' ( t_1 ) \right) ( t_2 - t_1 ) \, \mathrm {d}\tau \\&\le \psi ' ( t_1 ) ( t_2 - t_1 ) + \int _0^1 \left( \frac{\mathrm {e}^{\mu \tau \vert t_2 - t_1 \vert } - 1 }{ \mu \tau \vert t_2 - t_1 \vert } \right) \psi '' ( t_1 ) \tau ( t_2 - t_1 )^2 \, \mathrm {d}\tau \\&= \psi ' ( t_1 ) ( t_2 - t_1 ) + \frac{ \left( \mathrm {e}^{\mu \vert t_2 - t_1 \vert } - \mu \vert t_2 - t_1 \vert - 1 \right) }{\mu ^2} \psi '' ( t_1 ) \, ; \end{aligned}$$

similarly, we obtain

$$\begin{aligned} \psi ( t_2 ) - \psi ( t_1 ) \ge \psi ' ( t_1 ) ( t_2 - t_1 ) + \frac{ \left( \mathrm {e}^{- \mu \vert t_2 - t_1 \vert } + \mu \vert t_2 - t_1 \vert - 1 \right) }{\mu ^2} \psi '' ( t_1 ) \, . \end{aligned}$$

Lemma 3.3 follows from setting $\psi = \varphi$, $\mu = \varDelta$, $t_2 = 0$, and $t_1 = \alpha$.

Appendix D Proof of Proposition 3.3

Suppose that . We write

$$\begin{aligned} f ( \rho _k ) - f ( \rho _{k + 1} )&\ge - \tau {\langle { \nabla f ( \rho _{k} ), f ( \rho _{k + 1} ) - f ( \rho _{k} ) }\rangle } \\&\ge \tau \alpha _k^{-1} H ( \rho _{k + 1}, \rho _k ) \\&= \tau \alpha _k \alpha _k^{-2} H ( \rho _k ( \alpha _k ), \rho _k ) \\&\ge \tau {\underline{\alpha }} \kappa H ( \rho _k ( {\bar{\alpha }} ), \rho _k ) \\&\ge 0 , \end{aligned}$$

for large enough $k \in {\mathcal {K}}$, where the first inequality follows from the Armijo line search rule, the second follows from Lemma B.1, and the third follows from Corollary 3.1. Taking limit, we obtain that $H ( \rho _k ( {\bar{\alpha }} ), \rho _k ) \rightarrow 0$ as $k \rightarrow \infty$ in ${\mathcal {K}}$.

Suppose that . Let $( \alpha _k )_{k \in {\mathcal {K}}'}$, ${\mathcal {K}}' \subseteq {\mathcal {K}}$, be a subsequence converging to zero. According to the Armijo rule, we have

$$\begin{aligned} f ( \rho _k ( r^{-1} \alpha _k ) ) - f ( \rho _k ) > \tau {\langle { \nabla f ( \rho _k ), \rho _k ( r^{-1} \alpha _k ) - \rho _k ( \alpha _k )}\rangle } , \end{aligned}$$

(11)

for large enough $k \in {\mathcal {K}}$. The mean value theorem says that the left-hand side equals ${\langle { \nabla f ( \sigma ), \rho _k ( r^{-1} \alpha _k ) - \rho _k }\rangle }$ for some $\sigma$ in the line segment jointing $\rho _k ( r^{-1} \alpha _k )$ and $\rho _k$. Then, (11) can be equivalently written as

$$\begin{aligned}&{\langle { \nabla f ( \sigma ) - \nabla f ( \rho _k ), \rho _k ( r^{-1} \alpha _k ) - \rho _k }\rangle } \nonumber \\&\quad > - ( 1 - \tau ) {\langle { \nabla f ( \rho _k ), \rho _k ( r^{-1} \alpha _k ) - \rho _k ( \alpha _k ) }\rangle } . \end{aligned}$$

(12)

By Pinsker’s inequality and Hölder’s inequality, we obtain

$$\begin{aligned}&\Vert \nabla f ( \sigma ) - \nabla f ( \rho _k ) \Vert _\infty \sqrt{2 H ( \rho _k ( r^{-1} \alpha _k ), \rho _k )} \nonumber \\&\quad \ge \Vert \nabla f ( \sigma ) - \nabla f ( \rho _k ) \Vert _\infty \Vert \rho _k ( r^{-1} \alpha _k ), \rho _k \Vert _1 \nonumber \\&\quad \ge {\langle { \nabla f ( \sigma ) - \nabla f ( \rho _k ), \rho _k ( r^{-1} \alpha _k ) - \rho _k }\rangle } . \end{aligned}$$

(13)

for large enough $k \in {\mathcal {K}}$. Note that $r^{-1} \alpha _k \le {\bar{\alpha }}$ for large enough $k \in {\mathcal {K}}$. By Lemma B.1 and Corollary 3.2, we obtain

$$\begin{aligned}&- {\langle { \nabla f ( \rho _k ), \rho _k ( r^{-1} \alpha _k ) - \rho _k ( \alpha _k ) }\rangle } \ge \frac{H ( \rho _k ( r^{-1} \alpha _k ), \rho _k )}{r^{-1} \alpha _k } \nonumber \\&\ge \sqrt{ \kappa H ( \rho _k ( {\bar{\alpha }} ), \rho _k ) } \sqrt{ H ( \rho _k ( r^{-1} \alpha _k ), \rho _k ) } , \end{aligned}$$

(14)

for large enough $k \in {\mathcal {K}}$. Since $H ( \rho _k ( r^{-1} \alpha _k ), \rho _k )$ is strictly positive for all $k \in {\mathcal {K}}'$ by assumption, (12), (13), and (14) imply

$$\begin{aligned} \Vert \nabla f ( \sigma ) - \nabla f ( \rho _k ) \Vert _\infty > ( 1 - \tau ) \sqrt{ \frac{\kappa H ( \rho _k ( {\bar{\alpha }} ), \rho _k )}{2} } \ge 0 . \end{aligned}$$

Taking limits, we obtain that $H ( \rho _k ( {\bar{\alpha }} ), \rho _k ) \rightarrow 0$ a $k \rightarrow \infty$ in ${\mathcal {K}}'$.