Generalized $C(\alpha )$ Tests for Estimating Functions with Serial Dependence

Abstract

We propose generalized $C(\alpha )$ tests for testing linear and nonlinear parameter restrictions in models specified by estimating functions. The proposed procedures allow for general forms of serial dependence and heteroskedasticity, and can be implemented using any root-n consistent restricted estimator. The asymptotic distribution of the proposed statistic is established under weak regularity conditions. We show that earlier $C(\alpha )$-type statistics are included as special cases. The problem of testing hypotheses fixing a subvector of the complete parameter vector is discussed in detail as another special case. We also show that such tests provide a simple general solution to the problem of accounting for estimated parameters in the context of two-step procedures where a subvector of model parameters is estimated in a first step and then treated as fixed.

Appendix

Proof of Proposition 1

To simplify notation, we shall assume throughout that $\omega \in \mathscr {D} _{J}$ (an event with probability 1) and drop the symbol $\omega$ from the random variables considered. In order to obtain the asymptotic null distribution of the generalized $C(\alpha )$ statistic defined in (14), we first need to show that $P(\tilde{\theta }_{n}^{0})$ and $J_{n}(\tilde{\theta }_{n}^{0})$ converge in probability to $P(\theta _{0})$ and $J(\theta _{0})$ respectively. The consistency of $P(\tilde{\theta } _{n}^{0}),$ i.e.

$$\begin{aligned} \underset{n\rightarrow \infty }{\text {plim }}\big [P\big (\tilde{ \theta }_{n}^{0}\big )-P(\theta _{0})\big ]=0, \end{aligned}$$

(73)

follows simply from the consistency of $\tilde{\theta }_{n}^{0}$ [Assumption 9] and the continuity of $P(\theta )$ at $\theta _{0}$ [Assumption 7]. Further, by Assumption 8, since $P(\theta )$ is continuous in open neighborhood of $\theta _{0},$ we also have

$$\begin{aligned} \text {rank }[\tilde{P}_{n}]=\text {rank }[P(\theta _{0})]=p_{1}. \end{aligned}$$

(74)

Consider now $J_{n}(\tilde{\theta }_{n}^{0}).$ By Assumption 5, for any $\varepsilon >0$ and $\varepsilon _{1}>0,$ we can choose $\delta _{1} :=\delta (\varepsilon _{1}, \varepsilon )>0$ and a positive integer $n_{1}(\varepsilon , \delta _{1})$ such that: (i) $U_{J}(\delta _{1}, \varepsilon , \theta _{0})\le \varepsilon _{1}/2,$ and (ii) $n>n_{1}(\varepsilon , \delta _{1})$ entails

$$\begin{aligned} \mathsf {P}\big [ \varDelta _{n}(\theta _{0}, \delta )>\varepsilon \big ] = \mathsf {P}\big [\{\omega :\varDelta _{n}(\theta _{0}, \delta , \omega )>\varepsilon \}\big ]\le U_{J}(\delta _{1}, \varepsilon , \theta _{0})\le \varepsilon _{1}/2. \end{aligned}$$

Further, by the consistency of $\tilde{\theta }_{n}^{0}$ [Assumption 9], we can choose $n_{2}(\varepsilon _{1}, \delta _{1})$ such that $n>n_{2}(\varepsilon _{1}, \delta _{1})$ entails $\mathsf {P}[\Vert \tilde{\theta }_{n}^{0}-\theta _{0}\Vert \le \delta _{1}]\ge 1-(\varepsilon _{1}/2).$ Then, using the Boole-Bonferroni inequality, we have for $n>\max \{n_{1}(\varepsilon , \delta _{1}), n_{2}(\varepsilon _{1}, \delta _{1})\}$:

$$\begin{aligned} \begin{array}{ll} \mathsf {P}\big [\big \Vert J_{n}\big (\tilde{\theta }_{n}^{0}\big )-J(\theta _{0})\big \Vert \le \varepsilon \big ] &{} \ge \mathsf {P}\big [\big \Vert \tilde{\theta }_{n}^{0}-\theta _{0}\big \Vert \le \delta _{1}\;\text {and}\;\big \Vert J_{n}\big (\tilde{\theta }_{n}^{0}\big )-J(\theta _{0})\big \Vert \le \varepsilon \big ] \\ &{} \ge \mathsf {P}\big [\big \Vert \tilde{\theta }_{n}^{0}-\theta _{0}\big \Vert \le \delta _{1}\;\text {and}\;\varDelta _{n}\big ( \theta _{0}, \delta _{1}\big ) \le \varepsilon \big ] \\ &{} \ge 1-\mathsf {P}\big [\big \Vert \tilde{\theta }_{n}^{0}-\theta _{0}\big \Vert>\delta _{1}\big ]-\mathsf {P}\big [\varDelta _{n}\big ( \theta _{0}, \delta _{1}\big ) >\varepsilon \big ] \\ &{} \ge 1-(\varepsilon _{1}/2)-(\varepsilon _{1}/2)=1-\varepsilon _{1}. \end{array} \end{aligned}$$

Thus,

$$\begin{aligned} \underset{n\rightarrow \infty }{\lim \inf }{} \, \mathsf {P}\big [\big \Vert J_{n}\big (\tilde{\theta }_{n}^{0}\big )-J(\theta _{0})\big \Vert \le \varepsilon \big ]\ge 1-\varepsilon _{1},\quad \text {for all }\varepsilon>0,\;\varepsilon _{1}>0, \end{aligned}$$

hence

$$\begin{aligned} \underset{n\rightarrow \infty }{\lim }\,\mathsf {P}\big [\big \Vert J_{n}\big (\tilde{ \theta }_{n}^{0}\big )-J(\theta _{0})\big \Vert \le \varepsilon \big ]=1,\quad \text {for all }\varepsilon >0, \end{aligned}$$

(75)

or, equivalently,

$$\begin{aligned} \underset{n\rightarrow \infty }{\text {plim}} \,\big [J_{n}\big (\tilde{\theta }_{n}^{0}\big )-J (\theta _{0})\big ]=0. \end{aligned}$$

(76)

By Assumption 4, we can write [setting $0/0=0]:$

$$\begin{aligned} \big \Vert \sqrt{n}\,\big [D_{n}\big (\tilde{\theta }_{n}^{0}\big )-D_{n}(\theta _{0})\big ] -J(\theta _{0})\sqrt{n}\,\big (\tilde{\theta }_{n}^{0}-\theta _{0}\big )\big \Vert =\sqrt{n} \,\big \Vert R_{n}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert \\ =\frac{\big \Vert R_{n}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert }{\big \Vert \tilde{ \theta }_{n}^{0}-\theta _{0}\big \Vert }\sqrt{n}\,\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \end{aligned}$$

where

$$\begin{aligned} \frac{\big \Vert R_{n}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert }{\big \Vert \tilde{ \theta }_{n}^{0}-\theta _{0}\big \Vert }\le r_{n}(\delta , \theta _{0})\;\text { when }\tilde{\theta }_{n}^{0}\in N_{0}\;\text {and }\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \le \delta \end{aligned}$$

and $\mathop {\lim \sup }\nolimits _{n\rightarrow \infty }\,\mathsf {P}\big [ r_{n}(\delta , \theta _{0})>\varepsilon \big ] <U_{D}(\delta , \varepsilon , \theta _{0}).$ Thus, for any $\varepsilon >0$ and $\delta >0,$ we have:

$$\begin{aligned} \mathsf {P}\big [ \frac{\big \Vert R_{n}(\tilde{\theta }_{n}^{0}, \theta _{0})\big \Vert }{\big \Vert \tilde{\theta }_{n}^{0}-\theta _{0}\big \Vert }\le \varepsilon \big ]\ge & {} \mathsf {P}\big [r_{n}(\delta , \theta _{0})\le \varepsilon ,\tilde{\theta }_{n}^{0}\in N_{0}\;\text {and }\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \le \delta \big ] \\\ge & {} 1-\mathsf {P}\big [r_{n}(\delta , \theta _{0})>\varepsilon \,\big ]- \mathsf {P}\big [\,\tilde{\theta }_{n}^{0}\notin N_{0}\;\text {or }\big \Vert \tilde{ \theta }_{n}^{0}-\theta _{0}\big \Vert >\delta \big ] \end{aligned}$$

hence, using the consistency of $\tilde{\theta }_{n}^{0},$

$$\begin{aligned} \begin{array}{ll} \underset{n\rightarrow \infty }{\lim \inf }\,\mathsf {P}\big [ \big \Vert R_{n}\big ( \tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert /\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \le \varepsilon \big ] &{} \ge 1-\underset{ n\rightarrow \infty }{\,\lim \sup }\,\mathsf {P}\big [ r_{n}(\delta , \theta _{0})>\varepsilon \,\big ] \\ &{} \quad -\underset{n\rightarrow \infty }{\,\lim \sup }\,\mathsf {P}\big [ \,\tilde{\theta }_{n}^{0}\notin N_{0}\;\text {or }\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert >\delta \big ] \\ &{} \ge 1-U_{D}(\delta , \varepsilon , \theta _{0}). \end{array} \end{aligned}$$

Since $\mathop {\lim }\nolimits _{\delta \downarrow 0}\,U_{D}(\delta , \varepsilon , \theta _{0})=0,$ it follows that $\mathop {\lim }\nolimits _{n\rightarrow \infty }\, \mathsf {P}[\Vert R_{n}(\tilde{\theta }_{n}^{0}, \theta _{0})\Vert /\Vert \tilde{\theta }_{n}^{0}-\theta _{0}\Vert \le \varepsilon ]=1$ for any $\varepsilon >0,$ or equivalently,

$$\begin{aligned} \big \Vert R_{n}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert /\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \underset{n\rightarrow \infty }{\overset{\mathsf {p} }{\longrightarrow }}0. \end{aligned}$$

Since $\sqrt{n}\,(\tilde{\theta }_{n}^{0}-\theta _{0})$ is asymptotically bounded in probability (by Assumption 9), this entails:

$$\begin{aligned} \sqrt{n}\,\big \Vert R_{n}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert =\frac{ \big \Vert R_{n}(\tilde{\theta }_{n}^{0}, \theta _{0})\big \Vert }{\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert }\sqrt{n}\,\big \Vert \tilde{\theta }_{n}^{0}-\theta _{0}\big \Vert \underset{n\rightarrow \infty }{\overset{\mathsf {p}}{ \longrightarrow }}0\, \end{aligned}$$

(77)

and

$$\begin{aligned} \big \Vert \sqrt{n}\,\big [D_{n}\big (\tilde{\theta }_{n}^{0}\big )-D_{n}(\theta _{0})\big ]-J(\theta _{0})\sqrt{n}\,\big (\tilde{\theta }_{n}^{0}-\theta _{0}\big )\big \Vert \underset{ n\rightarrow \infty }{\overset{\mathsf {p}}{\longrightarrow }}0. \end{aligned}$$

(78)

By Taylor’s theorem and Assumptions 7 and 8, we also have the expansion:

$$\begin{aligned} \psi (\theta )=\psi (\theta _{0})+P(\theta _{0})(\theta -\theta _{0})+R_{2}(\theta , \theta _{0}), \end{aligned}$$

(79)

for $\theta \in N\subseteq N_{0}\cap V_{0},$ where N is a non-empty open neighborhood of $\theta _{0}$ and

$$\begin{aligned} \underset{\theta \rightarrow \theta _{0}}{\lim }\big \Vert R_{2}(\theta , \theta _{0})\big \Vert /\big \Vert \theta -\,\theta _{0}\big \Vert =0, \end{aligned}$$

i.e., $R_{2}(\theta , \theta _{0})=o(\big \Vert \theta -\,\theta _{0}\big \Vert )$, so that, using Assumption 10,

$$\begin{aligned} \sqrt{n}\,P(\theta _{0})\big (\tilde{\theta }_{n}^{0}-\theta _{0}\big )= & {} \sqrt{n} \,\big [\psi \big (\tilde{\theta }_{n}^{0}\big )-\psi (\theta _{0})\big ]-\sqrt{n}\,R_{2}\big (\tilde{ \theta }_{n}^{0}, \theta _{0}\big ) \nonumber \\= & {} -\sqrt{n}\,R_{2}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big ) \end{aligned}$$

(80)

for $\tilde{\theta }_{n}^{0}\in N,$ and

$$\begin{aligned} \big \Vert \sqrt{n}\,P(\theta _{0})\big (\tilde{\theta }_{n}^{0}-\theta _{0}\big )\big \Vert = \frac{\big \Vert R_{2}\big (\tilde{\theta }_{n}^{0}, \theta _{0}\big )\big \Vert }{\big \Vert \tilde{ \theta }_{n}^{0}-\theta _{0}\big \Vert }\sqrt{n}\big \Vert \tilde{\theta } _{n}^{0}-\theta _{0}\big \Vert \underset{n\rightarrow \infty }{\overset{\mathsf {p} }{\longrightarrow }}0. \end{aligned}$$

(81)

By (74) and (76) jointly with the Assumptions 3, 6, 7, 8, 11 and 12, we have:

$$\begin{aligned} \text {rank }[\tilde{P}_{n}]=p_{1},\text { rank }[\tilde{J}_{n}]=p,\text { rank }[\tilde{I}_{n}]=m,\text { rank } [W_{n}]=m\text {,} \end{aligned}$$

(82)

so the matrices $\tilde{J}_{n},\;\tilde{I}_{n},$ and $W_{n}$ all have full column rank. Since $\mathop {\text {plim}}\limits _{n\rightarrow \infty }\, \tilde{P}_{n}=P(\theta _{0})$ and $\mathop {\text {plim}}\limits _{n\rightarrow \infty } \tilde{J}_{n}=J(\theta _{0})$, we can then write:

$$\begin{aligned} \underset{n\rightarrow \infty }{\text {plim}}\,\big [\tilde{J} _{n}^{\,\prime }\,W_{n}\,\tilde{J}_{n}\big ]^{-1}= & {} \big [ J(\theta _{0})^{\prime }W_{0}\,J(\theta _{0})\big ] ^{-1},\;\underset{n\rightarrow \infty }{\text {plim}} \,\tilde{Q}_{n}=Q(\theta _{0}), \\ \underset{n\rightarrow \infty }{\text {plim}} \,\tilde{Q}_{n}\, \tilde{J}_{n}= & {} \underset{n\rightarrow \infty }{\text {plim}}\, \tilde{Q}_{n}\,J(\theta _{0})=Q(\theta _{0})\,J(\theta _{0})=P(\theta _{0}), \end{aligned}$$

where $\tilde{Q}_{n} :=\tilde{Q}[W_{n}]=\tilde{P}_{n}[\tilde{J} _{n}^{\,\prime } W_{n} \tilde{J}_{n}]^{-1}\tilde{J} _{n}^{\,\prime } W_{n}.$ Then, using (78) and (81), it follows that:

$$\begin{aligned} \begin{array}{ll} \underset{n\rightarrow \infty }{\text {plim}} &{} \left\{ \sqrt{n}\, \tilde{Q}_{n}\,D_{n}\big (\tilde{\theta }_{n}^{0}\big )-\sqrt{n}Q(\theta _{0})\,D_{n}(\theta _{0})\right\} \\ &{} =\,\underset{n\rightarrow \infty }{\text {plim}}\left\{ \sqrt{n} \,\tilde{Q}_{n}\,D_{n}\big (\tilde{\theta }_{n}^{0}\big )-Q(\theta _{0})\sqrt{n} \,D_{n}(\theta _{0})\right\} -\,\underset{n\rightarrow \infty }{\text {plim}} \left\{ P(\theta _{0})\sqrt{n}\,(\tilde{\theta } _{n}^{0}-\theta _{0})\right\} \\ &{} =\,\underset{n\rightarrow \infty }{\text {plim}}\quad \left\{ \tilde{Q} _{n}\big [\sqrt{n}\,[D_{n}\big (\tilde{\theta }_{n}^{0}\big )-D_{n}(\theta _{0})]-J(\theta _{0})\sqrt{n}\,(\tilde{\theta }_{n}^{0}-\theta _{0})\big ] \right\} \\ &{} \quad \quad +\;\underset{n\rightarrow \infty }{\text {plim}}\quad \left\{ \big [\tilde{Q}_{n}-Q(\theta _{0})\big ]\sqrt{n} \,D_{n}(\theta _{0})+\big [\tilde{Q}_{n}\,J(\theta _{0})-P(\theta _{0})\big ] \sqrt{n}\,(\tilde{\theta }_{n}^{0}-\theta _{0})\right\} \\ &{} =\,\underset{n\rightarrow \infty }{\text {plim}}\quad \left\{ \tilde{Q} _{n}\big [\sqrt{n}\,[D_{n}\big (\tilde{\theta }_{n}^{0}\big )-D_{n}(\theta _{0})]-J(\theta _{0})\sqrt{n}\,(\tilde{\theta }_{n}^{0}-\theta _{0})\big ] \right\} =0. \end{array} \end{aligned}$$

We conclude that the asymptotic distribution of $\sqrt{n}\,\tilde{Q}_{n}D_{n}(\tilde{\theta }_{n}^{0})$ is the same as the one of $Q(\theta _{0})\sqrt{n}\,D_{n}(\theta _{0}),$ namely (by Assumption 2) a $\mathrm {N}[0,V_{\psi }(\theta _{0})]$ distribution where

$$\begin{aligned} V_{\psi }(\theta )=Q(\theta )\,I(\theta )\,Q(\theta )^{\prime } \end{aligned}$$

and $V_{\psi }(\theta _{0})$ has rank $p_{1}=\,\mathrm {rank}\big [ Q(\theta _{0})\big ] =\,\mathrm {rank}[P(\theta _{0})].$ Consequently, the estimator

$$\begin{aligned} \tilde{V}_{\psi }\big (\tilde{\theta }_{n}^{0}\big )=\tilde{Q}_{n}\,\tilde{I}_{n}\, \tilde{Q}_{n}^{\,\prime } \end{aligned}$$

(83)

converges to $V_{\psi }(\theta _{0})$ in probability and, by (82),

$$\begin{aligned} \text { rank }\,\big [\tilde{V}_{\psi }\big (\tilde{\theta }_{n}^{0}\big )\big ] =p_{1}. \end{aligned}$$

(84)

Thus the test criterion

$$\begin{aligned} {PC}\big (\tilde{\theta }_{n}^{0}; \psi , W_{n}\big )=nD_{n}\big (\tilde{\theta } _{n}^{0}; Z_{n}\big )^{\prime }\tilde{Q}\big [ W_{n}\big ] ^{\prime }\left\{ \tilde{Q}\big [ W_{n}\big ] \tilde{I}_{n}\,\tilde{Q}\big [ W_{n}\big ] ^{\prime }\right\} ^{-1}\tilde{Q}[ W_{n}] D_{n}\big (\tilde{\theta } _{n}^{0}; Z_{n}\big )\, \end{aligned}$$

has an asymptotic $\chi ^{2}(p_{1})$ distribution.$\square$

Proof of Proposition 2

Consider the (non-empty) open neighborhood $N=N_{1}\cap N_{2}$ of $\theta _{0}.$ For any $\theta \in N$ and $\omega \in \mathscr {Z},$ we can write

$$\begin{aligned} \big \Vert J(\theta )-J(\theta _{0})\big \Vert\le & {} \big \Vert J_{n}(\theta , \omega )-J(\theta )\big \Vert +\big \Vert J_{n}(\theta _{0}, \omega )-J(\theta _{0})\big \Vert \\&+\;\big \Vert J_{n}(\theta , \omega )-J_{n}(\theta _{0}, \omega )\big \Vert \\\le & {} 2\;\underset{\theta \in N}{\sup }\big \Vert J_{n}(\theta , \omega )-J(\theta )\big \Vert +\big \Vert J_{n}(\theta , \omega )-J_{n}(\theta _{0}, \omega )\big \Vert \end{aligned}$$

By Assumption 14b, we have

$$\begin{aligned} \underset{n\rightarrow \infty }{\text {plim}}\quad \Big (\,\underset{ \theta \in N}{\sup }\big \Vert J_{n}(\theta , \omega )-J(\theta )\big \Vert \Big )\le \,\underset{n\rightarrow \infty }{\text {plim}} \Big (\,\underset{\theta \in N_{2}}{\sup }\big \Vert J_{n}(\theta , \omega )-J(\theta )\big \Vert \Big )=0 \end{aligned}$$

and we can find a subsequence $\left\{ J_{n_{t}}(\theta ,\omega ):t{=}1,2,\dots \right\}$ of $\left\{ J_{n}(\theta ,\omega ):n=1,2,\dots \right\}$ such that

$$\begin{aligned} \underset{\theta \in N}{\sup }\left\{ \big \Vert J_{n_{t}}(\theta , \omega )-J(\theta )\big \Vert \right\} \underset{t\rightarrow \infty }{ \longrightarrow }0\quad a.s. \end{aligned}$$

Let

$$\begin{aligned} CS=\left\{ \omega \in \mathscr {Z}:\,\underset{t\rightarrow \infty }{\text {lim}} \Big (\,\underset{\theta \in N}{\sup }\big \Vert J_{n_{t}}(\theta , \omega )-J(\theta )\big \Vert \Big )=0\right\} \end{aligned}$$

and $\varepsilon >0.$ By definition, $\mathsf {P}\big [ \omega \in CS\big ] =1.$ For $\,\omega \in CS,$ we can choose $t_{0}(\varepsilon , \omega )$ such that

$$\begin{aligned} t\ge t_{0}(\varepsilon ,\omega )\Rightarrow 2\;\underset{\theta \in N}{\sup }\left\{ \big \Vert J_{n_{t}}(\theta , \omega )-J(\theta )\big \Vert \right\} <\varepsilon /2. \end{aligned}$$

Further, since $J_{n}(\theta , \omega )$ is continuous in $\theta$ at $\theta _{0},$ we can find $\delta (n, \omega )>0$ such that

$$\begin{aligned} \big \Vert \theta -\theta _{0}\big \Vert<\delta (n,\omega )\Rightarrow \big \Vert J_{n}(\theta , \omega )-J_{n}(\theta _{0}, \omega )\big \Vert <\varepsilon /2. \end{aligned}$$

Thus, taking $t_{0}=t_{0}(\varepsilon , \omega )$ and $n=n_{t_{0}},$ we find that $\big \Vert \theta -\theta _{0}\big \Vert <\delta (n_{t_{0}},\omega )$ implies

$$\begin{aligned} \big \Vert J(\theta )-J(\theta _{0})\big \Vert <\dfrac{\varepsilon }{2}+ \dfrac{\varepsilon }{2}=\varepsilon . \end{aligned}$$

In other words, for any $\varepsilon >0,$ we can choose $\delta =\delta (n_{t_{0}},\varepsilon )>0$ such that

$$\begin{aligned} \big \Vert \theta -\theta _{0}\big \Vert<\delta \Rightarrow \big \Vert J(\theta )-J(\theta _{0})\big \Vert <\varepsilon , \end{aligned}$$

and the function $J(\theta )$ must be continuous at $\theta _{0}.$ Part (a) of the Proposition is established.

Set $\overline{\varDelta }_{n}(N_{2}, \omega ) :=\sup \left\{ \Vert J_{n}(\theta , \omega )-J(\theta )\Vert :\theta \in N_{2}\right\} .$ To get Assumption 5, we note that

$$\begin{aligned} \varDelta _{n}(\theta _{0}, \delta , \omega )\,:= & {} \sup \left\{ \Vert J_{n}(\theta , \omega )-J(\theta _{0})\Vert :\theta \in N_{2}\;\text {and } 0\le \big \Vert \theta -\,\theta _{0}\big \Vert \le \delta \right\} \\\le & {} \overline{\varDelta }_{n}(N_{2}, \omega )\, \end{aligned}$$

for any $\delta >0,$ hence, by Assumption 14b,

$$\begin{aligned} \underset{n\rightarrow \infty }{\lim \sup }\;\mathsf {P}\big [\{\omega :\varDelta _{n}(\theta _{0}, \delta , \omega )>\varepsilon \}\big ]\le & {} \,\underset{n\rightarrow \infty }{\lim \sup }\;\mathsf {P}\big [ \left\{ \omega :\overline{\varDelta }_{n}(N_{2}, \omega )>\varepsilon \right\} \big ] \\\le & {} U_{J}(\delta , \varepsilon , \theta _{0}) \end{aligned}$$

for any function $U_{J}(\delta , \varepsilon , \theta _{0})$ that satisfies the conditions of Assumption 5. The latter thus holds with $V_{0}$ any non-empty open neighborhood of $\theta _{0}$ such that $V_{0}\subseteq N_{2}$. To obtain Assumption 4, we note that Assumption 14 entails $D_{n}(\theta , \omega )$ is continuously differentiable in an open neighborhood of $\theta _{0}$ for all $\omega \in \mathscr {D}_{J},$ so that we can apply Taylor’s formula for a function of several variables (see Edwards [26, Section II.7]) to each component of $D_{n}(\theta , \omega ):$ for all $\theta$ in an open neighborhood U of $\theta _{0}$ (with $U\subseteq N_{2}$), we can write

$$\begin{aligned} D_{in}(\theta , \omega )= & {} D_{in}(\theta _{0}, \omega )+J_{n}\big (\bar{ \theta }_{n}^{\,i}(\omega ), \omega \big )_{i\cdot }(\theta -\,\theta _{0}) \\= & {} D_{in}(\theta _{0}, \omega )+J(\theta _{0})_{i\cdot }(\theta -\,\theta _{0})+R_{in}\big (\bar{\theta }_{n}^{\,i}(\omega ), \theta _{0}, \omega \big ) ,\;i=1, \ldots , m, \end{aligned}$$

where $J_{n}(\theta , \omega )_{i\cdot }$ and $J(\theta )_{i\cdot }$ are the i-th rows of $J_{n}(\theta , \omega )$ and $J(\theta )$ respectively,

$$\begin{aligned} R_{in}\big (\bar{\theta }_{n}^{\,i}(\omega ), \theta _{0}, \omega \big )=\big [ J_{n}\big (\bar{\theta }_{n}^{\,i}(\omega ), \omega \big )_{i\cdot }-J(\theta _{0})_{i\cdot }\big ](\theta -\,\theta _{0}) \end{aligned}$$

and $\bar{\theta }_{n}^{\,i}(\omega )$ belongs to the line joining $\theta$ and $\theta _{0}.$ Further, for $\theta \in U,$

$$\begin{aligned} \big |R_{in}\big (\bar{\theta }_{n}^{\,i}(\omega ), \theta _{0}, \omega \big ) \big |\le & {} \big \Vert J_{n}\big (\bar{\theta }_{n}^{\,i}(\omega ), \omega \big )_{i\cdot }-J(\theta _{0})_{i\cdot }\big \Vert \,\big \Vert \theta -\,\theta _{0}\big \Vert \\\le & {} \big \Vert J_{n}\big (\bar{\theta }_{n}^{\,i}(\omega ), \omega \big )-J(\theta _{0})\big \Vert \,\big \Vert \theta -\,\theta _{0}\big \Vert \\\le & {} \big \Vert \theta -\,\theta _{0}\big \Vert \,\sup \left\{ \Vert J_{n}(\theta , \omega )-J(\theta )\Vert :\theta \in N_{2}\right\} ,\;i=1, \ldots , m, \end{aligned}$$

hence, on defining $N_{0}=U,$

$$\begin{aligned} R_{n}(\theta , \theta _{0}, \omega )=\big [R_{1n}\big (\bar{\theta } _{n}^{\,1}(\omega ), \theta _{0}, \omega \big ), \ldots , R_{mn}\big ( \bar{\theta }_{n}^{\,m}(\omega ), \theta _{0}, \omega \big )\big ]^{\prime }, \end{aligned}$$

we see that

$$\begin{aligned} \big \Vert R_{n}(\theta , \theta _{0}, \omega )\big \Vert\le & {} \overset{ m}{\underset{1=1}{\,\sum }}\big |R_{in}\big (\bar{\theta }_{n}^{\,i}(\omega ), \theta _{0}, \omega \big )\big | \\\le & {} m\,\big \Vert \theta -\,\theta _{0}\big \Vert \,\underset{\theta \in N_{2}}{\sup }\left\{ \Vert J_{n}(\theta , \omega )-J(\theta )\Vert \right\} \, \end{aligned}$$

and

$$\begin{aligned} r_{n}(\delta , \theta _{0}, \omega )\,&:=\sup \left\{ \frac{\big \Vert R_{n}(\theta , \theta _{0}, \omega )\big \Vert }{\big \Vert \theta -\,\theta _{0}\big \Vert }:\theta \in N_{0}\;\text { and }0<\big \Vert \theta -\,\theta _{0}\big \Vert \le \delta \right\} \\\le & {} m\,\sup \left\{ \Vert J_{n}(\theta , \omega )-J(\theta )\Vert :\theta \in N_{2}\right\} \, \end{aligned}$$

Thus $r_{n}(\delta , \theta _{0}, \omega )\underset{n\rightarrow \infty }{ \overset{\mathsf {p}}{\longrightarrow }}0$ and

$$\begin{aligned} \underset{n\rightarrow \infty }{\lim \sup }\;\mathsf {P}\big [\{\omega :r_{n}(\delta , \theta _{0}, \omega )>\varepsilon \}\big ]\le U_{D}(\delta , \varepsilon , \theta _{0}) \end{aligned}$$

(85)

must hold for any function that satisfies the conditions of Assumption 4. This completes the proof.$\square$