Abstract
Given repeated observations of several subjects over time, i.e. a longitudinal data set, this paper introduces a new model to learn a classification of the shapes progression in an unsupervised setting: we automatically cluster a longitudinal data set in different classes without labels. Our method learns for each cluster an average shape trajectory (or representative curve) and its variance in space and time. Representative trajectories are built as the combination of pieces of curves. This mixture model is flexible enough to handle independent trajectories for each cluster as well as fork and merge scenarios. The estimation of such non linear mixture models in high dimension is known to be difficult because of the trapping states effect that hampers the optimisation of cluster assignments during training. We address this issue by using a tempered version of the stochastic EM algorithm. Finally, we apply our algorithm on different data sets. First, synthetic data are used to show that a tempered scheme achieves better convergence. We then apply our method to different real data sets: 1D RECIST score used to monitor tumors growth, 3D facial expressions and meshes of the hippocampus. In particular, we show how the method can be used to test different scenarios of hippocampus atrophy in ageing by using an heteregenous population of normal ageing individuals and mild cognitive impaired subjects.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdelkader, M. F., Abd-Almageed, W., Srivastava, A., & Chellappa, R. (2011). Silhouette-based gesture and action recognition via modeling trajectories on Riemannian shape manifolds. Computer Vision and Image Understanding., 3, 439–455. https://doi.org/10.1016/j.cviu.2010.10.006.
Allassonnière, A., & Chevallier, J. (2019). A new class of em algorithms. Escaping local minima and handling intractable sampling
Allassonniere, S., Chevallier, J., & Oudard, S. (2017). Learning spatiotemporal piecewise-geodesic trajectories from longitudinal manifold-valued data. In Advances in neural information processing systems (pp. 1152–1160).
Allassonnière, A., Durrleman, S., & Kuhn, E. (2015). Bayesian mixed effect atlas estimation with a diffeomorphic deformation model. SIAM Journal on Imaging Sciences, 8(3), 1367–1395.
Allassonnière, A., & Kuhn, E. (2010). Stochastic algorithm for bayesian mixture effect template estimation. ESAIM: Probability and Statistics, 14, 382–408.
Allassonnière, A., Kuhn, E., Trouvé, A., et al. (2010). Construction of bayesian deformable models via a stochastic approximation algorithm: a convergence study. Bernoulli, 16(3), 641–678.
Bône, A., Colliot, O., & Durrleman, S. (2018). Learning distributions of shape trajectories from longitudinal datasets: A hierarchical model on a manifold of diffeomorphisms. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 9271–9280).
Chakraborty, R., Singh, V., Adluru, N., & Vemuri, B. C. (2017). A geometric framework for statistical analysis of trajectories with distinct temporal spans. In Proceedings of the IEEE international conference on computer vision (pp. 172–181)
Charon, N., & Trouvé, A. (2013). The varifold representation of nonoriented shapes for diffeomorphic registration. SIAM Journal on Imaging Sciences, 6(4), 2547–2580.
Debavelaere, V., Bône, A., Durrleman, S., & Allassonnière, S. (2019) . Clustering of longitudinal shape data sets using mixture of separate or branching trajectories
Delyon, B., Lavielle, M., Moulines, E., et al. (1999). Convergence of a stochastic approximation version of the em algorithm. The Annals of Statistics, 27(1), 94–128.
Donohue, M. C., Jacqmin-Gadda, H., Le Goff, M., Thomas, R. G., Raman, R., Gamst, A. C., et al. (2014). Estimating long-term multivariate progression from short-term data. Alzheimer’s & Dementia, 10(5), S400–S410.
Durrleman, S., Allassonnière, A., & Joshi, S. (2013). Sparse adaptive parameterization of variability in image ensembles. International Journal of Computer Vision, 101(1), 161–183.
Fletcher, P. T. (2013). Geodesic regression and the theory of least squares on riemannian manifolds. International Journal of Computer vision, 105(2), 171–185.
Hong, Y., Singh, N., Kwitt, R., & Niethammer, M. (2015). Group testing for longitudinal data. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 9123, pp. 139–151). Springer. https://doi.org/10.1007/978-3-319-19992-4_11
Jedynak, B. M., Lang, A., Liu, B., Katz, E., Zhang, Y., Wyman, B. T., et al. (2012). A computational neurodegenerative disease progression score: Method and results with the alzheimer’s disease neuroimaging initiative cohort. Neuroimage, 63(3), 1478–1486.
Kendall, D. G. (1984). Shape manifolds, procrustean metrics, and complex projective spaces. Bulletin of the London Mathematical Society, 16(2), 81–121. https://doi.org/10.1112/blms/16.2.81.
Kim, H. J., Adluru, N., Suri, H., Vemuri, B. C., Johnson, S., & Singh, V. (2017). Riemannian nonlinear mixed effects models: Analyzing longitudinal deformations in neuroimaging. In Proceedings—30th IEEE conference on computer vision and pattern recognition (CVPR 2017) (Vol. 2017-Janua, pp. 5777–5786). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/CVPR.2017.612; ISBN: 9781538604571.
Lorenzen, P., Davis, B. C., & Joshi, S. (2005). Unbiased atlas formation via large deformations metric mapping. In International conference on medical image computing and computer-assisted intervention (pp. 411–418). Springer.
Lorenzi, M., Ayache, N., & Pennec, X. (2011). Schild’s ladder for the parallel transport of deformations in time series of images. In Biennial international conference on information processing in medical imaging (pp. 463–474). Springer
Louis, M., Bône, A., Charlier, B., Durrleman, S. Alzheimer’s Disease Neuroimaging Initiative, et al. (2017). Parallel transport in shape analysis: a scalable numerical scheme. In International conference on geometric science of information (pp. 29–37). Springer
Miller, M. I., Trouvé, A., & Younes, L. (2006). Geodesic shooting for computational anatomy. Journal of Mathematical Imaging and Vision, 24(2), 209–228.
Muralidharan P., & Fletcher, P. T. (2012). Sasaki metrics for analysis of longitudinal data on manifolds. In 2012 IEEE conference on computer vision and pattern recognition (pp. 1027–1034). IEEE
Schiratti, J.-B., Allassonniere, S., Colliot, O., & Durrleman, S. (2015). Learning spatiotemporal trajectories from manifold-valued longitudinal data. In Advances in neural information processing systems (pp. 2404–2412).
Schiratti, J.-B., Allassonnière, A., Colliot, O., & Durrleman, S. (2017). A bayesian mixed-effects model to learn trajectories of changes from repeated manifold-valued observations. The Journal of Machine Learning Research, 18(1), 4840–4872.
Singh, N., Hinkle, J., Joshi, S., & Fletcher, P. T. (2016). Hierarchical geodesic models in diffeomorphisms. International Journal of Computer Vision, 117(1), 70–92.
Srivastava, A., Joshi, S. H., Mio, W., & Liu, X. (2005). Statistical shape analysis: Clustering, learning, and testing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4), 590–602. https://doi.org/10.1109/TPAMI.2005.86.
Su, J., Kurtek, S., Klassen, E., Srivastava, A., et al. (2014). Statistical analysis of trajectories on Riemannian manifolds: bird migration, hurricane tracking and video surveillance. The Annals of Applied Statistics, 8(1), 530–552.
Therasse, P., Arbuck, S. G., Eisenhauer, E. A., Wanders, J., Kaplan, R. S., Rubinstein, L., et al. (2000). New guidelines to evaluate the response to treatment in solid tumors. Journal of the National Cancer Institute, 92(3), 205–216.
Vaillant, M., & Glaunès, J., (2005). Surface matching via currents. In Biennial international conference on information processing in medical imaging (pp. 381–392). Springer
Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009). Diffeomorphic demons: efficient non-parametric image registration. NeuroImage, 45(1 Suppl), S61–S72. https://doi.org/10.1016/j.neuroimage.2008.10.040.
Yin, L., Chenand X., Sun, Y., Worm, T., & Reale, M. (2008). A high-resolution 3d dynamic facial expression database, 2008. In IEEE international conference on automatic face and gesture recognition, Amsterdam, The Netherlands (Vol. 126).
Acknowledgements
This work has been partly funded by the European Research Council with grant 678304.
Author information
Authors and Affiliations
Consortia
Corresponding author
Additional information
Communicated by B. C. Vemuri.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Data used in preparation of this article were obtained from the Alzheimers Disease Neuroimaging Initiative (ADNI) database. As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: https://adni.loni.usc.edu.
Rights and permissions
About this article
Cite this article
Debavelaere, V., Durrleman, S., Allassonnière, S. et al. Learning the Clustering of Longitudinal Shape Data Sets into a Mixture of Independent or Branching Trajectories. Int J Comput Vis 128, 2794–2809 (2020). https://doi.org/10.1007/s11263-020-01337-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-020-01337-8