Abstract
There is stunning rapid development of human brains in the first year of life. Some studies have revealed the tight connection between cognition skills and cortical morphology in this period. Nonetheless, it is still a great challenge to predict cognitive scores using brain morphological features, given issues like small sample size and missing data in longitudinal studies. In this work, for the first time, we introduce the path signature method to explore hidden analytical and geometric properties of longitudinal cortical morphology features. A novel BrainPSNet is proposed with a differentiable temporal path signature layer to produce informative representations of different time points and various temporal granules. Further, a two-stream neural network is included to combine groups of raw features and path signature features for predicting the cognitive score. More importantly, considering different influences of each brain region on the cognitive function, we design a learning-based attention mask generator to automatically weight regions correspondingly. Experiments are conducted on an in-house longitudinal dataset. By comparing with several recent algorithms, the proposed method achieves the state-of-the-art performance. The relationship between morphological features and cognitive abilities is also analyzed.
X. Zhang and J. Cheng—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adeli, E., Meng, Y., Li, G., Lin, W., Shen, D.: Multi-task prediction of infant cognitive scores from longitudinal incomplete neuroimaging data. NeuroImage 185, 783–792 (2019)
Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Advances in Neural Information Processing Systems, pp. 41–48 (2007)
Boedihardjo, H., Geng, X., Lyons, T., Yang, D.: The signature of a rough path: uniqueness. Adv. Math. 293, 720–737 (2016)
Bonnier, P., Kidger, P., Arribas, I.P., Salvi, C., Lyons, T.: Deep signatures. arXiv preprint arXiv:1905.08494 (2019)
Braaten, E.B.: Mullen scales of early learning. In: The SAGE Encyclopedia of Intellectual and Developmental Disorders, vol. 34, no. 4, pp. 379–382 (2018)
Chen, J., Zhou, J., Ye, J.: Integrating low-rank and group-sparse structures for robust multi-task learning. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 42–50 (2011)
Chen, K.T.: Integration of paths-a faithful representation of paths by noncommutative formal power Series. Trans. Am. Math. Soc. 89(2), 395 (1958)
Chevyrev, I., Kormilitzin, A.: A primer on the signature method in machine learning. arXiv preprint arXiv:1603.03788 (2016)
Girault, J.B., et al.: Cortical structure and cognition in infants and toddlers. Cereb. Cortex 30(2), 786–800 (2020)
Hambly, B., Lyons, T.: Uniqueness for the signature of a path of bounded variation and the reduced path group. Ann. Math. 171(1), 109–167 (2010)
Hazlett, H.C., et al.: Early brain development in infants at high risk for autism spectrum disorder. Nature 542(7641), 348–351 (2017)
Hopfinger, J.B., Buonocore, M.H., Mangun, G.R.: The neural mechanisms of top-down attentional control. Nat. Neurosci. 3(3), 284–291 (2000)
Jha, S.C., et al.: Environmental influences on infant cortical thickness and surface area. Cereb. Cortex 29(3), 1139–1149 (2018)
Ji, S., Ye, J.: An accelerated gradient method for trace norm minimization. In: Proceedings of the 26th International Conference On Machine Learning (ICML), pp. 457–464 (2009)
Jung, R.E., Haier, R.J.: The parieto-frontal integration theory (P-FIT) of intelligence: converging neuroimaging evidence. Behav. Brain Sci. 30(2), 135–154 (2007)
Kagan, J., Herschkowitz, N., Herschkowitz, E.: A Young Mind in a Growing Brain. Psychology Press, New York (2006)
Lai, S., Jin, L., Yang, W.: Toward high-performance online HCCR: a CNN approach with dropdistortion, path signature and spatial stochastic max-pooling. Pattern Recogn. Lett. 89, 60–66 (2017)
Levin, D., Lyons, T., Ni, H.: Learning from the past, predicting the statistics for the future, learning an evolving system. arXiv preprint arXiv:1309.0260 (2013)
Li, C., Zhang, X., Liao, L., Jin, L., Yang, W.: Skeleton-based gesture recognition using several fully connected layers with path signature features and temporal transformer module. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8585–8593 (2019)
Li, G., Wang, L., Shi, F., Gilmore, J.H., Lin, W., Shen, D.: Construction of 4D high-definition cortical surface atlases of infants: methods and applications. Med. Image Anal. 25(1), 22–36 (2015)
Li, G., Wang, L., Shi, F., Lin, W., Shen, D.: Simultaneous and consistent labeling of longitudinal dynamic developing cortical surfaces in infants. Med. Image Anal. 18(8), 1274–1289 (2014)
Li, G., et al.: Mapping longitudinal development of local cortical gyrification in infants from birth to 2 years of age. Eur. J. Neurosci. 34(12), 4228–4238 (2014)
Li, G., et al.: Computational neuroanatomy of baby brains: a review. NeuroImage 185, 906–925 (2019)
Liu, M., Jin, L., Xie, Z.: PS-LSTM: capturing essential sequential online information with path signature and LSTM for writer identification. In: Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 664–669 (2017)
Lyall, A.E., et al.: Dynamic development of regional cortical thickness and surface area in early childhood. Cereb. Cortex 25(8), 2204–2212 (2015)
Lyons, T., Ni, H., Oberhauser, H.: A feature set for streams and an application to high-frequency financial tick data. In: ACM International Conference Proceeding Series, pp. 1–8 (2014)
Rekik, I., Li, G., Yap, P.T., Chen, G., Lin, W., Shen, D.: Joint prediction of longitudinal development of cortical surfaces and white matter fibers from neonatal MRI. NeuroImage 152, 411–424 (2017)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: International Conference on Computer Vision, pp. 618–626 (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (NIPS), pp. 5999–6009 (2017)
Wang, L., et al.: Volume-based analysis of 6-month-old infant brain MRI for autism biomarker identification and early diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 411–419. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_47
Wu, Z., Wang, L., Lin, W., Gilmore, J.H., Li, G., Shen, D.: Construction of 4D infant cortical surface atlases with sharp folding patterns via spherical patch-based group-wise sparse representation. Hum. Brain Mapp. 40(13), 3860–3880 (2019)
Yang, W., Lyons, T., Ni, H., Schmid, C., Jin, L., Chang, J.: Leveraging the path signature for skeleton-based human action recognition. arXiv preprint arXiv: 1707.03993 (2017)
Zhang, C., Adeli, E., Wu, Z., Li, G., Lin, W., Shen, D.: Infant brain development prediction with latent partial multi-view representation learning. IEEE Trans. Med. Imaging 38(4), 909–918 (2019)
Acknowledgement
XZ and XX are supported in part by the NSFC under grant U1801262; Guangzhou Key Laboratory of Body Data Science under grant 201605030011. HN is supported by the EPSRC under the program grant EP/S026347/1 and by the Alan Turing Institute under the EPSRC grant EP/N510129/1. GL is supported in part by NIH grant MH117943 and MH116225. LW is supported in part by NIH grant MH117943.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, X. et al. (2020). Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)
