Constrained distance based clustering for time-series: a comparative and experimental study

Abstract

Constrained clustering is becoming an increasingly popular approach in data mining. It offers a balance between the complexity of producing a formal definition of thematic classes—required by supervised methods—and unsupervised approaches, which ignore expert knowledge and intuition. Nevertheless, the application of constrained clustering to time-series analysis is relatively unknown. This is partly due to the unsuitability of the Euclidean distance metric, which is typically used in data mining, to time-series data. This article addresses this divide by presenting an exhaustive review of constrained clustering algorithms and by modifying publicly available implementations to use a more appropriate distance measure—dynamic time warping. It presents a comparative study, in which their performance is evaluated when applied to time-series. It is found that k-means based algorithms become computationally expensive and unstable under these modifications. Spectral approaches are easily applied and offer state-of-the-art performance, whereas declarative approaches are also easily applied and guarantee constraint satisfaction. An analysis of the results raises several influencing factors to an algorithm’s performance when constraints are introduced.

Appendices

Appendix A: Full metric scores

See Tables 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 and 18.

Table 8 Unconstrained mean ARI

Table 9 Mean consistency (measured using 50% constraint sets)

Table 10 Performance on ECG5000

Table 11 Performance on ElectricDevices

Table 12 Performance on FacesUCR

Table 13 Performance on InsectWingbeatSound

Table 14 Performance on MALLAT

Table 15 Performance on StarLightCurves

Table 16 Performance on TwoPatterns

Table 17 Performance on uWaveGestureLibraryX

Table 18 Performance on UWaveGestureLibraryAll

Appendix B: Constraint coherence

As described in Davidson et al. (2006): “We consider all constraint pairs composed of an ML and a CL constraint (pairs composed of the same constraint type cannot be contradictory). To determine the coherence of two constraints, a and b, we compute the projected overlap of each constraint on the other”.

Let $\mathbf {a}$ and $\mathbf {b}$ be vectors connecting the points constrained by a, i.e. $(a_1,a_2)$, and b, i.e. $(b_1,b_2)$, respectively. We first project the points bound by constraint a onto the line that is defined by the points bound by constraint b, such that

$$\begin{aligned} a'_1= & {} ((a_1 - b_1) \cdot \mathbf {e}) \mathbf {e} + b_1,\\ a'_2= & {} ((a_2 - b_1) \cdot \mathbf {e}) \mathbf {e} + b_1, \end{aligned}$$

where

$$\begin{aligned} \mathbf {e} = \frac{\mathbf {b}}{|\mathbf {b}|}. \end{aligned}$$

The points $a'_1$, $a'_2$, $b_1$, and $b_2$ now all exist in the 1D space described by the basis vector $\mathbf {e}$, and as such are projected into this 1D space, such that

$$\begin{aligned} a''_i = a'_i \mathbf {e}, \quad b''_i = b_i \mathbf {e}, \quad \text {where }\ i \in \{1,2\} . \end{aligned}$$

The 1D points of each constraint are then sorted such that $a''_1 \le a''_2$ and $b''_1 \le b''_2$. With this assumption satisfied, the overlap of constraint a on constraint b becomes

$$\begin{aligned} o_a^b = \max \left\{ 0, \min \{a''_2,b''_2\} - \max \{a''_1, b''_1\}\right\} . \end{aligned}$$

Two constraints are coherent if there is no overlap between them, such that

$$\begin{aligned} \text {coh}_{cm} = {\left\{ \begin{array}{ll} 1, \quad \text {if }\ o_c^m = 0\ \text { and }\ o_m^c = 0,\\ 0, \quad \text {otherwise,} \end{array}\right. } \end{aligned}$$

and the coherence of a set of constraints is defined to be the fraction of coherent constraints within the set, such that

$$\begin{aligned} \text {COH}(C) = \frac{\sum _{c \in C_{\text {CL}}, m \in C_{\text {ML}}}\text {coh}_{cm}}{|C_{\text {CL}}||C_{\text {ML}}|}. \end{aligned}$$