Skip to main content
SpringerLink
Log in

Adaptive temporal compressive sensing for video with motion estimation

  • Regular Paper
  • Published:
Optical Review Aims and scope Submit manuscript

Abstract

In this paper, we present an adaptive reconstruction method for temporal compressive imaging with pixel-wise exposure. The motion of objects is first estimated from interpolated images with a designed coding mask. With the help of motion estimation, image blocks are classified according to the degree of motion and reconstructed with the corresponding dictionary, which was trained beforehand. Both the simulation and experiment results show that the proposed method can obtain accurate motion information before reconstruction and efficiently reconstruct compressive video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., Nayar, S.K.: Video from a single coded exposure photograph using a learned over-complete dictionary. In: Proc. IEEE International Conference on Computer Vision: (ICCV), pp. 287–294, (2011)

  2. Liu, D., Gu, J., Hitomi, Y., Gupta, M., Mitsunaga, T., Nayar, S.K.: Efficient space-time sampling with pixel-wise coded exposure for high-speed imaging.” IEEE Trans. Pattern Anal. Mach. Intell. 36, 248–260 (2014)

    Article  Google Scholar 

  3. Llull, P., Liao, X., Yuan, X., Yang, J., Kittle, D., Carin, L., Sapiro, G., Brady, D.J.: Coded aperture compressive temporal imaging.” Opt. Express. 21, 10526–10545 (2013)

    Article  ADS  Google Scholar 

  4. Nagahara, H., Sonoda, T., Endo, K., Sugiyama, Y., Taniguchi, R.: High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. In Proc. International Conference on Computational Photography, pp. 1–11 (2016)

  5. Gao, L., Liang, J., Li, C., Wang, L.V.: Single-shot compressed ultrafast photography at one hundred billion frames per second.” Nature. 516, 74–77 (2014)

    Article  ADS  Google Scholar 

  6. Tang, C., Chen, Y., Feng, H., Xu, Z., Li, Q.: Motion deblurring based on local temporal compressive sensing for remote sensing image.” Opt. Eng. 55, 093106 (2016)

    Article  ADS  Google Scholar 

  7. Candes, E.J., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions.” Found. Comput. Math. 6, 227–254 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. Donoho, D.L.: Compressed sensing.” IEEE Trans. Inf. Theory. 52, 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Baraniuk, R.G., Goldstein, T., Sankaranarayanan, A.C., Studer, C., Veeraraghavan, A., Wakin, M.B.: Compressive video sensing: algorithms, architectures, and applications.” IEEE Signal Process. Mag. 34, 52–66 (2017)

    Article  ADS  Google Scholar 

  10. Iliadis, M., Spinoulas, L., Katsaggelos, A.K., et al.: Deep fully-connected networks for video compressive sensing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  11. Chen, Y., Tang, C., Feng, H., Xu, Z., Li, Q.: Adaptive reconstruction for coded aperture temporal compressive imaging.” Appl. Opt. 56(17), 4940–4947 (2017)

    Article  ADS  Google Scholar 

  12. Yuan, X., Yang, J., Llull, P., Liao, X., Sapiro, G., Brady, D.J., Carin, L.: Adaptive temporal compressive sensing for video. In: IEEE International Conference on Image Processing: (ICIP), pp. 14–18, (2013)

  13. Yang, J., Liao, X., Yuan, X., Llull, P., Brady, D.J., Sapiro, G., Carin, L.: Compressive sensing by learning a Gaussian mixture model from measurements.” IEEE Trans. Image Process. 24, 106–119 (2015)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  14. Bioucas-Dias, J.M., Figueiredo, M.A.: A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration.” IEEE Trans. Image Process. 16, 2992–3004 (2007)

    Article  ADS  MathSciNet  Google Scholar 

  15. Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit.” IEEE Trans. Inf. Theory. 53, 4655–4666 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Aharon, M., Elad, M., Bruckstein, A.: “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation.” IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  ADS  MATH  Google Scholar 

  17. Reddy, D., Veeraraghavan, A., Chellappa, R.: P2C2: programmable pixel compressive camera for high speed imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR),pp 329–336, (2011)

  18. Carin, L., Liu, D., Guo, B.: “Coherence, compressive sensing, and random sensor arrays.” IEEE Trans. Antenn. Propag. 53, 28–39 (2011)

    Article  ADS  Google Scholar 

  19. Duarte, M.F., Eldar, Y.C.: Structured compressed sensing: from theory to applications.” IEEE Trans. Signal Process. 59, 4053–4085 (2011)

    Article  ADS  MathSciNet  Google Scholar 

  20. Brox, T., et al.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Computer Vision, pp 25–36 ,(2004)

Download references

Acknowledgements

This work is supported by Fundamental Research Funds for the Central Universities and Space Innovation Fund Project, Jiangsu Science and Technology Program (BE2016119).

Author information

Authors and Affiliations

  1. State Key Lab of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, 310027, China

    Yeru Wang, Chaoying Tang, Yueting Chen, Huajun Feng, Zhihai Xu & Qi Li

Authors
  1. Yeru Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoying Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yueting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huajun Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yueting Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Tang, C., Chen, Y. et al. Adaptive temporal compressive sensing for video with motion estimation. Opt Rev 25, 215–226 (2018). https://doi.org/10.1007/s10043-018-0408-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10043-018-0408-5

Keywords

Search

Navigation