Skip to main content
SpringerLink
Log in

An improved vehicle panoramic image generation algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to reduce the traffic accidents caused by the blind area, vehicle panoramic view system has been paid more and more attention. However, the panoramic system is a complex and difficult system. In this paper, we propose an improved vehicle panoramic image generation algorithm. Several key technologies have been improved to ensure reliability and efficiency. First of all, we improve the spherical perspective projection algorithm (SPP) based on the scanning line idea and bilinear interpolation to rectification the fisheye image. Then the inverse perspective projection mapping of undistorted image is used to obtain a top view. In order to reduce computation, the method of manually selecting the target point is carried out. Finally, SURF algorithm is used to find the feature points between the bird’s-eye view images around vehicle. We further put forward to utilize a RANSAC algorithm based on block matching to eliminate the mismatched points in the key point matching process. Experimental results indicate that our vehicle panoramic image generation method works efficiently. The proposed algorithm can effectively remove the serious distortion of fisheye lens, and generate a panoramic image around the vehicle in the end. It possesses good robustness, and can be widely used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bawa VS, Kumar K, Kumar V (2017) A generalized non-linear method for distortion correction and top-down view conversion of fish eye images. Image Anal Stereol 36(2):141–150

    Article  Google Scholar 

  2. Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: European conference on computer vision, Springer, pp 404–417

  3. Cai C, Wang P, Liang Y-h (2016) Fast image stitching based on improved SURF. In: 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE, pp 411–416

  4. Dou J, Qin Q, Tu Z (2018) Robust image matching based on the information of SIFT. Optik 171:850–861

    Article  Google Scholar 

  5. Gao Y, Lin C, Zhao Y, Wang X, Wei S, Huang Q (2018) 3-D surround view for advanced driver assistance systems. IEEE Trans Intell Transp Syst 19(1):320–328

    Article  Google Scholar 

  6. He Y, Deng G, Wang Y, Wei L, Yang J, Li X, Zhang Y (2018) Optimization of SIFT algorithm for fast-image feature extraction in line-scanning ophthalmoscope. Optik 152:21–28

    Article  Google Scholar 

  7. Ji X, Zhang G, Chen X, Guo Q (2018) Multi-perspective tracking for intelligent vehicle. IEEE Trans Intell Transp Syst 19(2):518–529

    Article  Google Scholar 

  8. Lai S, Xiong Z, Chen L, Tan X, Zhang M (2014) Real-time distortion correction of fish-eye lens based on Bayer image signal. Opt Rev 21(2):162–173

    Article  Google Scholar 

  9. Lee J-H (2017) Panoramic image stitching using feature extracting and matching on embedded system. Trans Electr Electron Mater 18(5):273–278

    Google Scholar 

  10. Leutenegger S, Chli M, Siegwart R (2011) BRISK: binary robust invariant scalable keypoints. In: 2011 IEEE international conference on computer vision (ICCV) Ieee, pp 2548–2555

  11. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. In: Twenty-fourth international joint conference on artificial intelligence

  12. Liu Y, Zhang L, Nie L, Yan Y (2016) Rosenblum DS fortune teller: predicting your career path. In: Thirtieth AAAI conference on artificial intelligence

  13. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  14. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  15. Lu Y, Hua Z, Gao K, Xu T (2018) Multiperspective image stitching and regularization via hybrid structure warping. Comput Sci Eng 20(2):10–23

    Article  Google Scholar 

  16. Ma J, Zhao J, Tian J, Yuille AL, Tu Z (2014) Robust point matching via vector field consensus. IEEE Trans Image Process 23(4):1706–1721

    Article  MathSciNet  MATH  Google Scholar 

  17. Ma J, Qiu W, Zhao J, Ma Y, Yuille AL, Tu Z (2015) Robust L2E estimation of transformation for non-rigid registration. IEEE Trans Signal Process 63(5):1115–1129

    Article  MathSciNet  MATH  Google Scholar 

  18. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481

    Article  Google Scholar 

  19. Ma J, Zhao J, Jiang J, Zhou H, Guo X (2019) Locality preserving matching. Int J Comput Vis 127(5):512–531

    Article  MathSciNet  Google Scholar 

  20. NBoSo C (2017) China statistical yearbook. China Statistics Press, Beijing

    Google Scholar 

  21. Qiang D, Jinghong L, Chao W, Qianfei Z (2017) Image mosaic algorithm based on improved BRISK. J Electron Inf Technol 39(2):444–450

    Google Scholar 

  22. Rashid M, Khan MA, Sharif M, Raza M, Sarfraz MM, Afza F (2018) Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimed Tools Appl:1–27

  23. Rublee E, Rabaud V, Konolige K, Bradski GR (2011) ORB: an efficient alternative to SIFT or SURF. In: 2011 international conference on computer vision, pp. 2564–2571

  24. Tiwari U, Mani U, Paul S, Nath N (2015) Non-linear method used for distortion correction of fish-eye lens: comparative analysis of different mapping functions. In: 2015 international conference on man and machine interfacing (MAMI), IEEE, pp 1–5

  25. Tu B, Liu L, Liu Y, Jin Y, Tang J (2013) High precision two-step calibration method for the fish-eye camera. Appl Opt 52(7):C37–C42

    Article  Google Scholar 

  26. Turturici M, Saponara S, Fanucci L, Franchi E (2014) Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications. J Real-Time Image Proc 9(3):463–478

    Article  Google Scholar 

  27. Victoria BA, Jorge RSA (2017) Manuel PHL SIFT-SURF commutation using fuzzy logic to image mosaicking. In: 2017 14th international conference on electrical engineering, computing science and automatic control (CCE). IEEE, pp 1–6

  28. Vourvoulakis J, Kalomiros J, Lygouras J (2017) FPGA accelerator for real-time SIFT matching with RANSAC support. Microprocess Microsyst 49:105–116

    Article  Google Scholar 

  29. Vourvoulakis J, Kalomiros J, Lygouras J (2018) FPGA-based architecture of a real-time SIFT matcher and RANSAC algorithm for robotic vision applications. Multimed Tools Appl 77(8):9393–9415. https://doi.org/10.1007/s11042-017-5042-x

    Article  Google Scholar 

  30. Wu J, Huang F, Hu W, He W, Tu B, Guo L, Ou X, Zhang G (2019) Study of multiple moving targets’ detection in fisheye video based on the moving blob model. Multimed Tools Appl 78(1):877–896. https://doi.org/10.1007/s11042-018-5763-5

    Article  Google Scholar 

  31. Xiong Z, Ying J, Zhang R (2011) Research of bird's-eye panoramic view for vehicle parking. 2011 International Conference on Multimedia Technology, IEEE:456–459

  32. Yan W, Hou C, Lei J, Fang Y, Gu Z, Ling N (2017) Stereoscopic image stitching based on a hybrid warping model. IEEE Transactions on Circuits and Systems for Video Technology 27(9):1934–1946

    Article  Google Scholar 

  33. Yang Z, Zhao Y, Hu X, Yin Y, Zhou L, Tao D (2018) A flexible vehicle surround view camera system by central-around coordinate mapping model. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6744-4

  34. Yeh Y-T, Peng C-K, Chen K-W, Chen Y-S, Hung Y-P (2014) Driver assistance system providing an intuitive perspective view of vehicle surrounding. In: Asian conference on computer vision. Springer, pp 403–417

  35. Yin X, Zhang J, Wu X, Huang J, Xu Y, Zhu L (2018) An improved lane departure warning algorithm based on fusion of F-Kalman filter and F-TLC. Multimed Tools Appl:1–20. https://doi.org/10.1007/s11042-018-6762-2

  36. Ying X-H, Hu Z-Y (2003) Fisheye lense distortion correction using spherical perspective projection constraint. Chinese journal of computers-chinese edition 26(12):1702–1708

    Google Scholar 

  37. Zeng Q, Wang Y, Liu J, Liu S (2017) A matching algorithm for large viewpoint changes images. Optik 137:268–278

    Article  Google Scholar 

  38. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22

  39. Zhang Y, Zhou S (2017) Study on methods for fish-eye image correction based on spherical projection model. In: 2017 5th international conference on Frontiers of manufacturing science and measuring technology (FMSMT 2017). Atlantis Press, Paris, pp 848–854

    Google Scholar 

  40. Zhang B, Appia V, Pekkucuksen I, Liu Y, Umit Batur A, Shastry P, Liu S, Sivasankaran S, Chitnis K (2014) A surround view camera solution for embedded systems. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 662–667

  41. Zhang J, Chen G, Jia Z (2017) An image stitching algorithm based on histogram matching and SIFT algorithm. Int J Pattern Recognit Artif Intell 31(04):1754006

    Article  Google Scholar 

  42. Zhou W, Liu Y, Lyu C, Zhou W, Peng J, Yang R, Shang H (2016) Real-time implementation of panoramic mosaic camera based on FPGA. In: 2016 IEEE International Conference on Real-time Computing and Robotics (RCAR), 2016 IEEE, pp 204–209

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (Grant No. 2017YFB0102500); the Natural Science Foundation of Jilin province (Grant No. 20170101133JC); the Korea Foundation for Advanced Studies’ International Scholar Exchange Fellowship for the academic year of 2017-2018, and Jilin University (Grant No. 5157050847, 2017XYB252).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Jindong Zhang, Xuelong Yin & Tong Liu

  2. Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Jindong Zhang

  3. State Key Laboratory of Automobile Simulation and Control, Jilin University, Changchun, 130025, China

    Jindong Zhang

  4. College of Software, Jilin University, Changchun, 130012, China

    Jing Luan

Authors
  1. Jindong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuelong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jindong Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Yin, X., Luan, J. et al. An improved vehicle panoramic image generation algorithm. Multimed Tools Appl 78, 27663–27682 (2019). https://doi.org/10.1007/s11042-019-07890-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07890-w

Keywords

Search

Navigation