Regrets of proximal method of multipliers for online non-convex optimization with long term constraints

Abstract

The online optimization problem with non-convex loss functions over a closed convex set, coupled with a set of inequality (possibly non-convex) constraints is a challenging online learning problem. A proximal method of multipliers with quadratic approximations (named as OPMM) is presented to solve this online non-convex optimization with long term constraints. Regrets of the violation of Karush-Kuhn-Tucker conditions of OPMM for solving online non-convex optimization problems are analyzed. Under mild conditions, it is shown that this algorithm exhibits ${{\mathcal {O}}}(T^{-1/8})$ Lagrangian gradient violation regret, ${{\mathcal {O}}}(T^{-1/8})$ constraint violation regret and ${{\mathcal {O}}}(T^{-1/4})$ complementarity residual regret if parameters in the algorithm are properly chosen, where T denotes the number of time periods. For the case that the objective is a convex quadratic function, we demonstrate that the regret of the objective reduction can be established even the feasible set is non-convex. For the case when the constraint functions are convex, if the solution of the subproblem in OPMM is obtained by solving its dual, OPMM is proved to be an implementable projection method for solving the online non-convex optimization problem.

Appendix A: Proof of Lemma 6

Proof

Let

$$\begin{aligned} r= \frac{\zeta }{4t_0\delta _{\max }^2}, \quad \rho =1- \frac{\zeta ^2}{8\delta _{\max }^2}, \end{aligned}$$

then it yields that $\rho =1- \frac{rt_0}{2}\zeta .$ Define $\eta (t)=Z_{t+t_0}-Z_t$, then we have $|\eta (t)|\le t_0 \delta _{\max }$ and hence

$$\begin{aligned} |r \eta (t)|\le \frac{\zeta }{4t_0\delta _{\max }^2} \cdot t_0 \delta _{\max } = \frac{\zeta }{4 \delta _{\max }} \le 1. \end{aligned}$$

(A1)

From (A1) and the following inequality

$$\begin{aligned} e^{\tau } \le 1+\tau +2 \tau ^2 \text{ when } |\tau |<1, \end{aligned}$$

we obtain

$$\begin{aligned} e^{rZ_{t+t_0}}= & {} e^{rZ_t}e^{r\eta (t)}\\\le & {} e^{rZ_t} [1+r \eta (t)+2r^2 t_0^2\delta _{\max }^2]\\= & {} e^{rZ_t} [1+r \eta (t)+r t_0\zeta /2]. \end{aligned}$$

Case 1: $Z_t\ge \theta$. In this case, one has from (13) that $\eta (t) \le -t_0 \zeta$ and hence

$$\begin{aligned} e^{rZ_{t+t_0}}\le & {} e^{rZ_t} [1-rt_0 \zeta +r t_0\zeta /2] \nonumber \\= & {} e^{rZ_t} [1-rt_0 \zeta /2] \nonumber \\= & {} \rho e^{rZ_t}. \end{aligned}$$

(A2)

Case 2: $Z_t< \theta$. In this case, one has from (13) that $\eta (t) \le t_0 \delta _{\max }$ and hence

$$\begin{aligned} e^{rZ_{t+t_0}}= & {} e^{rZ_t}e^{r \eta (t)} \nonumber \\\le & {} e^{rZ_t}e^{r t_0 \delta _{\max }} \nonumber \\\le & {} e^{r\theta }e^{r t_0 \delta _{\max }}. \end{aligned}$$

(A3)

Combining (A2) and (A3), we obtain

$$\begin{aligned} e^{rZ_{t+t_0}} \le \rho e^{rZ_t}+e^{r\theta }e^{r t_0 \delta _{\max }}. \end{aligned}$$

(A4)

We next prove the following inequality by induction,

$$\begin{aligned} e^{rZ_t} \le \frac{1}{1-\rho }e^{r\theta }e^{r t_0 \delta _{\max }},\ t\in \{0,1,\ldots \}. \end{aligned}$$

(A5)

We first consider the case $t \in \{0,1,\ldots , t_0\}$. From $|Z_{t+1}-Z_t|\le \delta _{\max }$ and $Z_0=0$ we have $Z_t \le t \delta _{\max }$. This, together with the fact that $\frac{e^{r\theta }}{1-\rho }\ge 1$, implies

$$\begin{aligned} e^{rZ_t} \le e^{rt\delta _{\max }}\le e^{r t_0\delta _{\max }} \le e^{r t_0\delta _{\max }} \frac{e^{r\theta }}{1-\rho }. \end{aligned}$$

Hence, (A5) is satisfied for all $t \in \{0,1,\ldots , t_0\}$. We now assume that (A5) holds true for all $t \in \{t_0+1,\ldots , \tau \}$ with arbitrary $\tau > t_0$. Consider $t=\tau +1$. By (A4), we have

$$\begin{aligned}&e^{rZ_{\tau +1}}\le \rho e^{rZ_{\tau +1-t_0}}+e^{r\theta }e^{r t_0 \delta _{\max }} \\&\quad \le \rho e^{r t_0\delta _{\max }} \frac{e^{r\theta }}{1-\rho }+e^{r\theta }e^{r t_0 \delta _{\max }} = e^{r t_0\delta _{\max }} \frac{e^{r\theta }}{1-\rho }. \end{aligned}$$

Therefore, the inequality (A5) holds for all $t\in \{0,1,\ldots \}$. Taking logarithm on both sides of (A5) and dividing by r yields

$$\begin{aligned} Z_t \le \theta +t_0 \delta _{\max } + \frac{1}{r} \log \left( \frac{1}{1-\rho } \right) = \theta +t_0 \delta _{\max } + t_0 \frac{4 \delta _{\max }^2}{\zeta } \log \left( \frac{8\delta _{\max }^2}{\zeta ^2} \right) . \end{aligned}$$

The proof is completed. $\square$