Exploring the latent segmentation space for the assessment of multiple change-point models

Abstract

This paper addresses the retrospective or off-line multiple change-point detection problem. Multiple change-point models are here viewed as latent structure models and the focus is on inference concerning the latent segmentation space. Methods for exploring the space of possible segmentations of a sequence for a fixed number of change points may be divided into two categories: (i) enumeration of segmentations, (ii) summary of the possible segmentations in change-point or segment profiles. Concerning the first category, a dynamic programming algorithm for computing the top $N$ most probable segmentations is derived. Concerning the second category, a forward-backward dynamic programming algorithm and a smoothing-type forward-backward algorithm for computing two types of change-point and segment profiles are derived. The proposed methods are mainly useful for exploring the segmentation space for successive numbers of change points and provide a set of assessment tools for multiple change-point models that can be applied both in a non-Bayesian and a Bayesian framework. We show using examples that the proposed methods may help to compare alternative multiple change-point models (e.g. Gaussian model with piecewise constant variances or global variance), predict supplementary change points, highlight overestimation of the number of change points and summarize the uncertainty concerning the position of change points.

Appendices

Appendix A

Pseudo-code of the dynamic programming algorithm for computing the top $N$ most probable segmentations in $J$ segments

The following convention is adopted in the presentation of the pseudo-codes: The operator ‘$:=$’ denotes the assignment of a value to a variable (or the initialization of a variable with a value). The expression $( t<T-1\;?\;J-2:J-1)$ is a shorthand for if $t<T-1$ then $J-2$ else $J-1$. This convention applies to the three Appendices.

The working variables SegmentLogLikelihood$(u)$, Length$_{j} ^{n}(t)$ and Rank$_{j}^{n}(t)$ are introduced for this implementation. For each time $t$, each segment $j$ and each rank $n$, two backpointers must be recorded, the first Length$_{j}^{n}(t)$ giving the optimal length of the segment and the second Rank$_{j}^{n}(t)$ giving the associated rank.

Forward recursion

In a first step, the log-likelihoods for segments ending at time $t$, SegmentLogLikelihood$(u)$ are computed for each segment length. Then in a second step, the quantities $\alpha _{j}^{n}( t)$ are computed for each segment $j$ and each rank $n$. The increasing number of segmentations for small values of $t$ is taken into account by initializing at $-\infty$ $\alpha _{j}^{n}(t)$ for each segment $j$ and for $n=N_{j}(t)+1,\ldots ,N$. The quantities SegmentLogLikelihood$(u)$ should be stored for each time $u$ while the quantities $\alpha _{j}^{n}(t)$ and the backpointers Length$_{j}^{n}(t)$ and Rank$_{j}^{n}( t)$ should be stored for each time $t$, each segment $j$ and each rank $n$.

Backtracking

Appendix B

Pseudo-code of the forward-backward dynamic programming algorithm

The working variables SegmentLogLikelihood$(u)$ and Output$_{j}(t)$ are introduced for this implementation.

Forward recursion

In a first step, the log-likelihoods for segments ending at time $t$, SegmentLogLikelihood$(u)$ are computed for each segment length. Then in a second step, the quantities $\alpha _{j}(t)$ are computed for each segment $j$. The quantities SegmentLogLikelihood$(u)$ should be stored for each time $u$ while the quantities $\alpha _{j}(t)$ should be stored for each time $t$ and each segment $j$.

Backward recursion

In a first step, the log-likelihoods for segments beginning at time $t$, SegmentLogLikelihood$(u)$ are computed for each segment length. Then in a second step, the quantities $\beta _{j}(t)$ are computed for each segment $j$ and the quantities $\gamma _{j}(t)$ are extracted in the case of the computation of segment profiles. The quantities SegmentLogLikelihood$(u)$ should be stored for each time $u$ while the quantities $\beta _{j}(t)$ and Output$_{j}(t)$ should be stored for each time $t$ and each segment $j$.

Appendix C

Pseudo-code of the smoothing-type forward-backward algorithm

The working variables SegmentLikelihood$(u)$, Norm$(t)$, ForwardNorm$(t)$ and BackwardNorm$(t)$ are introduced for this implementation.

Forward recursion

The minimum segment index should be 0 instead of $\max \{0,J-(T-t) \}$ for the transdimensional generalization.

In a first step, the log-likelihoods for segments ending at time $t$, SegmentLikelihood$(u)$ are computed for each segment length; see Appendix B. Then in a second step, the quantities $F_{j}(t)$ are computed for each segment $j$. The variable SegmentNorm is used to compute $\sum _{v=u}^{t-1}\log N_{v}$ for each $u<t$. The quantities $\sum _{u=0}^{t}\log N_{u}$ are stored in the auxiliary quantity ForwardNorm$(t)$ for each time $t$ for use in the backward recursion. The quantities SegmentLikelihood$(u)$ should be stored for each time $u$ while the quantities $F_{j}(t)$ should be stored for each time $t$ and each segment $j$.

Backward recursion

The maximum segment index should be $J-1$ instead of $\min ( J-1,t)$ for the transdimensional generalization.

In a first step, the log-likelihoods for segments beginning at time $t$, SegmentLikelihood$(u)$ are computed for each segment length; see Appendix B. Then in a second step, the quantities $B_{j}(t)$ are computed for each segment $j$. The variable SegmentNorm is used to compute $\sum _{v=t+1}^{u}\log M_{v}$ for each $u>t$. The quantities SegmentLikelihood$(u)$ should be stored for each time $u$ while the quantities $B_{j}(t)$ and Output$_{j}(t)$ should be stored for each time $t$ and each segment $j$ and only the current quantity BackwardNorm$(t)$ should be stored.