Abstract
Reconstructing under-sampled k-space measurements in Compressed Sensing MRI (CS-MRI) is classically solved by minimizing a regularized least-squares cost function. In the absence of fully-sampled training data, this optimization approach can still be amortized via a neural network that minimizes the cost function over a dataset of under-sampled measurements. Here, a crucial design choice is the regularization function(s) and corresponding weight(s). In this paper, we introduce HyperRecon – a novel strategy of using a hypernetwork to generate the parameters of a main reconstruction network as a function of the regularization weight(s), resulting in a regularization-agnostic reconstruction model. At test time, for a given under-sampled image, our model can rapidly compute reconstructions with different amounts of regularization. We propose and empirically demonstrate an efficient and data-driven way of maximizing reconstruction performance given limited hypernetwork capacity. Our code will be made publicly available upon acceptance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this paper, we assume a single coil acquisition.
- 2.
For baselines, the \(18 \times 18\) grid was linearly interpolated to \(100 \times 100\) to match the hypernetwork landscapes.
References
Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: Model-based deep learning architecture for inverse problems. IEEE Trans. Med. 38(2), 394–405 (2019)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. 38(8), 1788–1800 (2019)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Img. Sci. 2(1), 183–202 (2009)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
Brock, A., Lim, T., Ritchie, J., Weston, N.: SMASH: one-shot model architecture search through hypernetworks. In: ICLR (2018)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40, 120–145 (2011)
Chang, O., Flokas, L., Lipson, H.: Principled weight initialization for hypernetworks. In: ICLR (2020)
Chauffert, N., Ciuciu, P., Weiss, P.: Variable density compressed sensing in MRI. theoretical vs heuristic sampling strategies. In: 2013 IEEE 10th ISBI, April 2013
Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing (2009)
Cremer, C., Li, X., Duvenaud, D.: Inference suboptimality in variational autoencoders (2018)
Dalca, A.V., Guttag, J., Sabuncu, M.R.: Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In: IEEE CVPR, June 2018
Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Pure Appl. Math. 57, 1413–1457 (2003)
Figueiredo, M.A.T., Nowak, R.D.: An FM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12(8), 906–916 (2003)
Geethanath, S., et al.: Compressed sensing MRI: a review. Crit. Rev. Biomed. Eng. 41(3), 183–204 (2013)
Gershman, S.J., Goodman, N.D.: Amortized inference in probabilistic reasoning. In: CogSci (2014)
Ha, D., Dai, A., Le, Q.V.: Hypernetworks (2016)
Hoopes, A., Hoffmann, M., Fischl, B., Guttag, J., Dalca, A.V.: Hypermorph: amortized hyperparameter learning for image registration. arXiv preprint arXiv:2101.01035 (2021)
Hu, Y., Jacob, M.: Higher degree total variation (HDTV) regularization for image recovery. IEEE Trans. Image Process. 21(5), 2559–2571 (2012)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2014)
Klocek, S., Maziarka, L., Wolczyk, M., Tabor, J., Nowak, J., Śmieja, M.: Hypernetwork functional image representation. Lecture Notes in Computer Science, pp. 496–510. (2019)
Krueger, D., Huang, C.W., Islam, R., Turner, R., Lacoste, A., Courville, A.: Bayesian hypernetworks (2018)
Lorraine, J., Duvenaud, D.: Stochastic hyperparameter optimization through hypernetworks (2018)
Luo, G.: A review of automatic selection methods for machine learning algorithms and hyper-parameter values. Netw. Model Anal. Health Inform. Bioinforma. 5, 18 (2016). https://doi.org/10.1007/s13721-016-0125-6
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58(6), 1182–1195 (2007)
Marino, J., Yue, Y., Mandt, S.: Iterative amortized inference. arXiv preprint arXiv:1807.09356 (2018)
Pan, Z., Liang, Y., Zhang, J., Yi, X., Yu, Y., Zheng, Y.: Hyperst-net: hypernetworks for spatio-temporal forecasting (2018)
Ravishankar, S., Ye, J.C., Fessler, J.A.: Image reconstruction: from sparsity to data-adaptive methods and machine learning. Proc. IEEE 108(1), 86–109 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlin. Phenomena 60(1–4), 259–268 (1992)
Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: a review of Bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)
Shen, F., Yan, S., Zeng, G.: Meta networks for neural style transfer (2017)
Shu, R., Bui, H.H., Zhao, S., Kochenderfer, M.J., Ermon, S.: Amortized inference regularization (2018)
Ukai, K., Matsubara, T., Uehara, K.: Hypernetwork-based implicit posterior estimation and model averaging of CNN. In: Zhu, J., Takeuchi, I. (eds.) Proceedings of The 10th Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 95, pp. 176–191. PMLR, November 2018
Wang, A.Q., Dalca, A.V., Sabuncu, M.R.: Neural network-based reconstruction in compressed sensing MRI without fully-sampled training data. In: Deeba, F., Johnson, P., Würfl, T., Ye, J.C. (eds.) Machine Learning for Medical Image Reconstruction, pp. 27–37. Springer International Publishing, Cham (2020)
Wang, S., et al.: Accelerating magnetic resonance imaging via deep learning. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 514–517 (2016)
Yang, G., et al.: Dagan: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imag. 37(6), 1310–1321 (2018)
Yang, Y., Sun, J., Li, H., Xu, Z.: Deep admm-net for compressive sensing MRI. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 29. Curran Associates, Inc. Barcelona (2016)
Ye, N., Roosta-Khorasani, F., Cui, T.: Optimization methods for inverse problems. In: Wood, D., de Gier, J., Praeger, C., Tao, T. (eds.) 2017 MATRIX Annals. MATRIX Book Series, vol. 2, pp. 121–140. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-04161-8_9
Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for \(l_1\)-minimization with applications to compressed sensing. SIAM J. Imag. Sci. 1(1), 143–168 (2008)
Yu, T., Zhu, H.: Hyperparameter optimization. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds.) Automated Machine Learning. The Springer Series on Challenges in Machine Learning. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05318-5_1
Zhang, C., Ren, M., Urtasun, R.: Graph hypernetworks for neural architecture search (2019)
Acknowledgements
This work was supported by NIH grants R01LM012719 (MS), R01AG053949 (MS), 1R01AG064027 (AD), the NSF NeuroNex grant 1707312 (MS), and the NSF CAREER 1748377 grant (MS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, A.Q., Dalca, A.V., Sabuncu, M.R. (2021). HyperRecon: Regularization-Agnostic CS-MRI Reconstruction with Hypernetworks. In: Haq, N., Johnson, P., Maier, A., Würfl, T., Yoo, J. (eds) Machine Learning for Medical Image Reconstruction. MLMIR 2021. Lecture Notes in Computer Science(), vol 12964. Springer, Cham. https://doi.org/10.1007/978-3-030-88552-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-88552-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88551-9
Online ISBN: 978-3-030-88552-6
eBook Packages: Computer ScienceComputer Science (R0)
