An objective Bayesian analysis of the change point problem

Abstract

The Bayesian literature on the change point problem deals with the inference of a change in the distribution of a set of time-ordered data based on a sample of fixed size. This is the so-called “retrospective or off-line” analysis of the change point problem. A related but different problem is that of the “sequential” change point detection, mainly analyzed from a frequentist viewpoint. While the former typically focuses on the estimation of the position in which the change point occurs, the latter is a testing problem which has a natural formulation as a Bayesian model selection problem. In this paper we provide such a Bayesian formulation, which generalizes previous formulations such as the well-known CUSUM stopping rule. We show that the conventional improper priors (also called non-informative, objective or default), cannot be used either for sequential detection of the change or for retrospective estimation. Then, we propose objective intrinsic prior distributions for the unknown model parameters. The normal and Poisson cases are studied in detail and examples with simulated and real data are provided.

Appendices

Appendix 1

Proof of Lemma 1.

For the models $M_0 :P\left( {x_1 |\theta } \right) = {{\theta ^{x_1 } } \mathord{\left/ {\vphantom {{\theta ^{x_1 } } {x_1 !}}} \right. \kern-\nulldelimiterspace} {x_1 !}}\,\exp \left\{ { - \theta } \right\},$ and $M_1 :\left\{ {P\left( {x_1 |\theta _1 } \right)P\left( {x_2 |\theta _2 } \right),\pi ^D \left( {\theta _1 ,\theta _2 } \right) = k\theta _1^{ - 1/2} \theta _2^{ - 1/2} } \right\},$ where θ is an arbitrary but fixed value, and k is an arbitrary positive constant. The minimal training sample is a pair of independent random variables X₁,X₂ such that under model M₁, X _i ∽ P(x _i |θ _i ), and under M₀, X _i ∽ P(x _i |θ), i=0,1. Then, simple calculations give

$$ B_{01}^N (x_1 ,x_2 ) = \frac{{\theta ^{x_1 + x_2 } \exp \{ - 2\theta \} }} {{k\Gamma (x_1 + 1/2)\Gamma (x_2 + 1/2)}}. $$

Furthermore,

$$ \begin{aligned} E_{x_1 ,x_2 |\theta _1 ,\theta _2 }^{M_1 } B_{01}^N (x_1 ,x_2 ) & = \frac{{\exp \{ - (\theta _1 + \theta _2 + 2\theta )\} }} {k}\sum\limits_{x_1 = 0}^\infty \,\frac{{(\theta \theta _1 )^{x_1 } }} {{\Gamma (x_1 + 1/2)x_1 !}} \\ & \quad \times \sum\limits_{x_2 = 0}^\infty \,\frac{{(\theta \theta _2 )^{x_2 } }} {{\Gamma (x_2 + 1/2)x_2 !}}. \\ \end{aligned} $$

Using the equality $\sum\nolimits_{x = 0}^\infty {{{\left( {\theta \;\theta _1 } \right)^{x_2 } } \mathord{\left/ {\vphantom {{\left( {\theta \;\theta _1 } \right)^{x_2 } } {\Gamma \left( {x + 1/2} \right)x!}}} \right. \kern-\nulldelimiterspace} {\Gamma \left( {x + 1/2} \right)x!}} = {{F_0^1 \left( {1/2,\theta \;\theta _1 } \right)} \mathord{\left/ {\vphantom {{F_0^1 \left( {1/2,\theta \;\theta _1 } \right)} {\Gamma \left( {1/2} \right)}}} \right. \kern-\nulldelimiterspace} {\Gamma \left( {1/2} \right)}}}$ and then substitution in Eq. 12, Lemma 1 follows.

Appendix 2

Proof of Lemma 2.

Consider the model

$$ M_0^ * :N(x|\theta ,\tau ^2 ), $$

and

$$ M_1 :\left\{ {N\left( {x|\mu _1 ,\sigma _1^2 } \right)N\left( {y|\mu _2 ,\sigma _2^2 } \right),\pi _1^N ({\varvec{\mu }},{\varvec{\sigma }}) = \frac{{c_1 }} {{\sigma _1 \sigma _2 }}} \right\}. $$

The minimal training sample is a random vector (X₁,X₂,Y₁,Y₂) with independent components such that under model M₁, X _i ~ N(x _i |μ ₁,σ ²₁ ), Y _i ~ N(y _i |μ ₂,σ ²₂ ), and under M ^*₀ , X _i , Y _i ~ N(x|θ ,τ ²), i=1,2. We recall that a minimal training sample is a random vector of minimal size for which the marginal density is greater than zero and finite (except for a null set with respect to the Lebergue measure). Then,

$$ B_{01}^N (x,y) = \frac{1} {{m_1 ({\mathbf{x}},{\mathbf{y}})}}\prod\limits_{i = 1}^2 \,N(x_i |\theta ,\tau ^2 )N(y_i |\theta ,\tau ^2 ), $$

where

$$ m_1 (x,y) = c_1 \frac{1} {{2^2 |x_1 - x_2 ||y_1 - y_2 |}}. $$

Therefore,

$$ \begin{aligned} \pi ^I ({\varvec{\mu }},{\varvec{\sigma }}|\theta ,\tau ) & = \frac{1} {{4\sigma _1^3 \sigma _2^3 \tau ^4 }} \\ & \quad \times \int {|x_1 - x_2 |\exp \left\{ { - d_x^2 \left( {\tau ^{ - 2} + \sigma _1^{ - 2} } \right) - \frac{{(m_x - \theta )^2 }} {{\tau ^2 }} - \frac{{(m_x - \mu _1 )^2 }} {{\sigma _1^2 }}} \right\}{\text{d}}x_1 {\text{d}}x_2 } \\ & \quad \times \int {|y_1 - y_2 |\exp \left\{ { - d_y^2 \left( {\tau ^{ - 2} + \sigma _2^{ - 2} } \right) - \frac{{(m_y - \theta )^2 }} {{\tau ^2 }} - \frac{{(m_y - \mu _2 )^2 }} {{\sigma _2^2 }}} \right\}{\text{d}}y_1 {\text{d}}y_2 } , \\ \end{aligned} $$

where

$$ d_x^2 = \frac{{(x_1 - x_2 )^2 }} {4},\quad m_x = \frac{{x_1 + x_2 }} {2}, $$

$$ d_y^2 = \frac{{(y_1 - y_2 )^2 }} {4},\quad m_x = \frac{{y_1 + y_2 }} {2}. $$

Changing to the new variables

$$ u_1 = x_1 - x_2 ,\quad v_1 = x_1 + x_2 , $$

$$ u_2 = y_1 - y_2 ,\quad v_2 = y_1 + y_2 , $$

the result in Lemma 1 follows.