Skip to main content
SpringerLink
Log in

Gradient-Based Edge Effects on Lane Marking Detection using a Deep Learning-Based Approach

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Lane detection is part of the advanced driver assistance system (ADAS) equipped in intelligent vehicles. The system provides the driver with significant geometric information of the road ahead. Numerous deep learning techniques have been employed in lane detection because of the simplicity, ease, and efficiency of these techniques in learning discriminative features from RGB (red, green, and blue) images. However, existing works have rarely considered detecting lane markings during bad weather conditions, which could reduce lane detection performance. Hence, this paper proposed a Fully Convolutional Network (FCN) model with RGB and Canny edge detection used as the model’s spatial input. The proposed platform was developed using two scenarios: FCN-RGB-edge and FCN-edge. The model development was divided into three stages, namely data acquisition, platform development, and benchmarking against existing methods and data. Both scenarios using the proposed method yielded a 4% improvement compared to the original FCN-RGB images (i.e., the previous method). The Canny edge detection method successfully extracted necessary information from the images and neglected the water drops in rainy conditions by treating them as noise. In summary, the proposed method has the potential to boost the performance of the ADAS system in detecting lane markings in rainy conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Zou, Q.; Jiang, H.; Dai, Q.; Yue, Y.; Chen, L.; Wang, Q.: robust lane detection from continuous driving scenes using deep neural networks. [cs.CV], pp. 1–14 (2019). arXiv:1903.02193V2

  2. Grove, K., et al.: Commercial motor vehicle driver performance with adaptive cruise control in adverse weather. Procedia Manuf. 3(Ahfe), 2777–2783 (2015). https://doi.org/10.1016/j.promfg.2015.07.717

    Article  Google Scholar 

  3. Zakir, U., et al.: Current collision mitigation technologies for advanced driver assistance systems—a survey. PERINTIS eJ. 6(2), 78–90 (2016). https://doi.org/10.1105/tpc.15.01050

    Article  Google Scholar 

  4. Eichelberger, A.H.; McCartt, A.T.: Toyota drivers’ experiences with dynamic radar cruise control, pre-collision system, and lane-keeping assist. J. Saf. Res. 56, 67–73 (2016). https://doi.org/10.1016/j.jsr.2015.12.002

    Article  Google Scholar 

  5. Rudin, N. S. A.; Mustafah, Y. M.; Abidin, Z. Z.; Cho, J.; Zaki, H. F. M.: Vision-based lane departure warning system, vol. 2, no. 2, pp. 166–176 (2018)

  6. Kaur, G.; Kumar, D.: Lane detection techniques: a review. Int. J. Comput. Appl. 112(10), 975–8887 (2015)

    Google Scholar 

  7. Guo, C.; Mita, S.; McAllester, D.: Lane detection and tracking in challenging environments based on a weighted graph and integrated cues. IEEE/RSJ 2010 Int. Conf. Intell. Robot. Syst. IROS 2010 Conf. Proc. (2010). https://doi.org/10.1109/IROS.2010.5650695

    Article  Google Scholar 

  8. Lin, C.W.; Tseng, D.C.; Wang, H.Y.: A robust lane detection and verification method for intelligent vehicles. 3rd Int. Symp. Intell. Inf. Technol. Appl. IITA 1, 521–524 (2009). https://doi.org/10.1109/IITA.2009.178

    Article  Google Scholar 

  9. Huval, B., et al.: An empirical evaluation of deep learning on highway driving, pp. 1–7 (2015). https://doi.org/10.1002/pi.4767

  10. Neven, D.; De Brabandere, B.; Georgoulis, S.; Proesmans, M.; Van Gool, L.: Towards end-to-end lane detection: an instance segmentation approach. (2018).https://doi.org/10.1109/CRV.2016.27

  11. Gopalan, R.; Hong, T.; Shneier, M.; Chellappa, R.: A learning approach towards detection and tracking of lane markings. Intell. Transp. Syst. IEEE Trans. 13(3), 1088–1098 (2012). https://doi.org/10.1109/TITS.2012.2184756

    Article  Google Scholar 

  12. Kim, J.G., Yoo, J.H., Koo, J.C.: Road and lane detection using stereo camera. Proc. 2018 IEEE Int. Conf. Big Data Smart Comput. BigComp 2018 (2018). https://doi.org/10.1109/BigComp.2018.00117

    Article  Google Scholar 

  13. Field, S.I.: An urban autodriving algorithm based on a sensor-weighted integration field with deep learning. Electronics 9, 1–18 (2020)

    Google Scholar 

  14. Zhou, S.; Jiang, Y.; Xi, J., Gong, J.; Xiong, G.; Chen, H.: A novel lane detection based on geometrical model and Gabor filter. IEEE Intell. Veh. Symp. Proc. (2010). https://doi.org/10.1109/IVS.2010.5548087

    Article  Google Scholar 

  15. Mariut, F.; Fosalau, C.; Petrisor, D.: Lane mark detection using Hough transform. EPE 2012 Proc. 2012 Int Conf. Expo. Electr. Power Eng. (2012). https://doi.org/10.1109/ICEPE.2012.6463823

    Article  Google Scholar 

  16. Liu, G.; Wörgötter, F.; Markelić, I.: Combining statistical hough transform and particle filter for robust lane detection and tracking. IEEE Intell. Veh. Symp. Proc. (2010). https://doi.org/10.1109/IVS.2010.5548021

    Article  Google Scholar 

  17. Teng, Z.; Kim, J.; Kang, D.: Real-time Lane detection by using multiple cues. Transform, pp. 2334–2337 (2010)

  18. Revilloud, M.; Gruyer, D.; Pollard, E.: An improved approach for robust road marking detection and tracking applied to multi-lane estimation. IEEE Intell. Veh. Symp. Proc. Iv, 783–790 (2013). https://doi.org/10.1109/IVS.2013.6629562

    Article  Google Scholar 

  19. Assidiq, A.A.M.; Khalifa, O.O.; Islam, M.R.; Khan, S.: Real time lane detection for autonomous vehicles. Proc. Int. Conf. Comput. Commun. Eng. ICCCE08 Glob Links Hum. Dev. (2008). https://doi.org/10.1109/ICCCE.2008.4580573

    Article  Google Scholar 

  20. Wang, Q.; Gao, J.; Yuan, Y.: Embedding structured contour and location prior in siamesed fully convolutional networks for road detection. IEEE Trans. Intell. Transp. Syst. 19(1), 230–241 (2018). https://doi.org/10.1109/TITS.2017.2749964

    Article  Google Scholar 

  21. Li, J.; Mei, X.; Prokhorov, D.; Tao, D.: Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans. Neural Networks Learn. Syst. 28(3), 690–703 (2017). https://doi.org/10.1109/TNNLS.2016.2522428

    Article  Google Scholar 

  22. Zhu, H.; Yuen, K.V.; Mihaylova, L.; Leung, H.: Overview of environment perception for intelligent vehicles. IEEE Trans. Intell. Transp. Syst. 18(10), 2584–2601 (2017). https://doi.org/10.1109/TITS.2017.2658662

    Article  Google Scholar 

  23. Ding, D.; Lee, C.; Lee, K.Y.: An adaptive road ROI determination algorithm for lane detection. IEEE Reg. 10 Annu. Int. Conf. Proc. TENCON (2013). https://doi.org/10.1109/TENCON.2013.6718807

    Article  Google Scholar 

  24. Zhang, W.; Liu, H.; Wu, X.; Xiao, L.; Qian, Y.; Fang, Z.: Lane marking detection and classification with combined deep neural network for driver assistance. Proc. Inst. Mech. Eng. Part D J. Automob. Eng. (2018). https://doi.org/10.1177/0954407018768659

    Article  Google Scholar 

  25. Daigavane, P.M.; Bajaj, P.R.: Road lane detection with improved canny edges using ant colony optimization. 2010 3rd Int. Conf. Emerg. Trends Eng. Technol. (2010). https://doi.org/10.1109/ICETET.2010.128

    Article  Google Scholar 

  26. Boggavarapu, H.D.V.J.K.M.L.N.P.; Vaddi, R.S.: A robust multi color lane marking detection approach for Indian scenario. Int. J. Adv. Comput. Sci. Appl. 2(5), 71–75 (2011)

    Google Scholar 

  27. Lee, C.; Moon, J.H.: Robust lane detection and tracking for real-time applications. IEEE Trans. Intell. Transp. Syst. 1, 1–6 (2018). https://doi.org/10.1109/TITS.2018.2791572

    Article  Google Scholar 

  28. Lin, Q.; Youngjoon, H.; Hahn, H.: Real-time lane detection based on extended edge-linking algorithm. 2nd Int. Conf. Comput. Res. Dev. ICCRD 2010 (2010). https://doi.org/10.1109/ICCRD.2010.166

    Article  Google Scholar 

  29. Burlachenko, K.: Lane detection using Fourier based line detector, pp. 1282–1285, (2013)

  30. Aly, M.: [Q先生提到的有趣应用] Real time detection of lane markers in urban streets.pdf. https://doi.org/10.1109/TITS.2017.2723575

  31. Zhang, X.; Chen, M.; Zhan, X.: A combined approach to single-camera-based lane detection in driverless navigation. 2018 IEEE ION Position Locat. Navig. Symp. PLANS 2018 Proc. (2018). https://doi.org/10.1109/PLANS.2018.8373484

    Article  Google Scholar 

  32. Sun, Z.; Bebis, G;, Miller, R.: On-road vehicle detection: a review. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 694–711 (2006). https://doi.org/10.1109/TPAMI.2006.104

    Article  Google Scholar 

  33. Badrinarayanan, V.; Kendall, A.; Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  34. Tian, Y., et al.: Lane marking detection via deep convolutional neural network. Neurocomputing 280, 46–55 (2017). https://doi.org/10.1016/j.neucom.2017.09.098

    Article  Google Scholar 

  35. Kim, J.; Kim, J.; Jang, G.J.; Lee, M.: Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Netw. 87, 109–121 (2017). https://doi.org/10.1016/j.neunet.2016.12.002

    Article  Google Scholar 

  36. Yu, M.; Member, S.; Liu, L.; Shao, L.; Member, S.: Structure-preserving binary representations for RGB-D action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38(8), 1651–1664 (2016). https://doi.org/10.1109/TPAMI.2015.2491925

    Article  Google Scholar 

  37. Álvarez, J.M.; López, A.M.; Gevers, T.; Lumbreras, F.: Combining priors, appearance, and context for road detection. IEEE Trans. Intell. Transp. Syst. 15(3), 1168–1178 (2014). https://doi.org/10.1109/TITS.2013.2295427

    Article  Google Scholar 

  38. Song, W.; Liu, L.; Zhou, X.; Wang, C.: Road detection algorithm of integrating region and edge information. Proc. Int. Conf. Artif. Intell. Robot Int. Conf. Autom. Control Robot. Eng. ICAIR-CACRE’16 (2016). https://doi.org/10.1145/2952744.2952749

    Article  Google Scholar 

  39. Zitnick, C.L.; Dollár, P.: Edge boxes: locating object proposals from edges. Lect. Notes Comput. Sci. Include. Subser Lect. Notes Artif. Intell. Lect. Notes Bioinform. 8693 LNCS(PART 5), 391–405 (2014). https://doi.org/10.1007/978-3-319-10602-1_26

    Article  Google Scholar 

  40. Liu, S.; Yang, J.; Huang, C.; Yang, M.H.: Multi-objective convolutional learning for face labelling. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (2015). https://doi.org/10.1109/CVPR.2015.7298967

    Article  Google Scholar 

  41. Sotelo, M.A.; Rodriguez, F.J.; Magdalena, L.: VIRTUOUS : vision-based road transportation for unmanned operation on urban like scenarios. IEEE Trans. Intell. Transp. Syst. 5(2), 69–83 (2004). https://doi.org/10.1109/TITS.2004.828175

    Article  Google Scholar 

  42. He, Y.; Wang, H.; Zhang, B.: Color-based road detection in urban traffic scenes, vol. 5, no. 4, pp. 309–318, (2004)

  43. Brust, C.-A.; Sickert, S.; Simon, M.; Rodner, E.; Denzler, J.: Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding, (2015). https://doi.org/10.5220/0005355105100517

  44. Estate, T. I.: e-CAM50 _ CUP031 _ MOD Camera Module Datasheet, no. December, pp. 1–13, (2017)

  45. Badrinarayanan, V.; Handa, A.; Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling, (2015). https://doi.org/10.1103/PhysRevX.5.041024

  46. Kendall, A.; Badrinarayanan, V.; Cipolla, R.: Bayesian SegNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding (2015). https://doi.org/10.13140/RG.2.1.2985.2407

  47. Long, J.; Shelhamer, E.: Fully convolutional networks for semantic segmentation. [cs.CV], p. 10 (2015). arXiv:1411.4038V2

  48. Caltagirone, L.; Scheidegger, S.; Svensson, L.; Wahde, M.: Fast LIDAR-based road detection using fully convolutional neural networks. (2017). arXiv:1703.3613

  49. Fritsch, J.; Tobias, K.; Geiger, A.: A new performance measure and evaluation benchmark for road detection algorithms. Intell. Transp. Syst., pp. 1693–1700 (2013)

  50. Mohan, R.: Deep deconvolutional networks for scene parsing, (2014). arXiv:1411.4101

  51. Mendes, T.; Fremont, V.; Wolf, D. F.: Exploiting fully convolutional neural networks for fast road detection. Proc. IEEE Int. Conf. Robot. Auto. (ICRA), Stock. Sweden, pp. 3174–3179. https://doi.org/10.1109/ICRA.2016.7487486

  52. Levi, D.: StixelNet : A deep convolutional network for obstacle detection and road segmentation. Br. Mach. Vis. Conf., pp. 1–12, (2015)

  53. Laddha, A.; Kocamaz, M. K.; Navarro-serment, L. E.: Map-supervised road detection. 2016 IEEE Intell. Veh. Symp., pp. 118–123 (2016)

  54. Chen, X.: 3D Object proposals for accurate object class detection. Conf. Neural Inf. Process. Syst., pp. 1–9, (2015)

  55. Xiao, L.; Dai, B.; Liu, D.; Hu, T.; Wu, T.: CRF based road detection with multi-sensor fusion. IEEE Intell. Veh. Symp. Iv, 192–198 (2015). https://doi.org/10.1109/IVS.2015.7225685

    Article  Google Scholar 

  56. Vitor G. B. et al.: A probabilistic distribution approach for the classification of urban roads in complex environments. Proc. IEEE Work. Int. Conf. Robot Autom. (2014)

  57. Munoz, D.; Bagnell, J. A.; Hebert, M.: Stacked hierarchical labeling. Proc. 11th Eur. Conf. Comput. Vis. (2010)

  58. Mendes, C.; Fremont, V.; Wolf, D.: Vision-based road detection using contextual blocks, (2015). arXiv:1509.01122

Download references

Acknowledgements

This work was supported in part by the CRG 8.1: AI Predictive Model for HVAC Operation under Grant Q. K130000.2443.04G73 and in part by ANCHOR under Grant 4B386.

Author information

Authors and Affiliations

  1. Centre for Artificial Intelligence & Robotics, Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, 54100, Kuala Lumpur, Malaysia

    Noor Jannah Zakaria, Mohd Ibrahim Shapiai, Hilman Fauzi & Hossamelden Mohamed Amin Elhawary

  2. Advanced Vehicle System Research Group, Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, 54100, Kuala Lumpur, Malaysia

    Wira Jazair Yahya, Mohd Azizi Abdul Rahman, Irfan Bahiuddin & Mohd Hatta Mohammed Ariff

  3. ASEAN NCAP Operational Unit, Malaysian Institute of Road Safety Research, 43000, Kajang, Selangor, Malaysia

    Khairil Anwar Abu Kassim

Authors
  1. Noor Jannah Zakaria
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohd Ibrahim Shapiai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hilman Fauzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hossamelden Mohamed Amin Elhawary
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wira Jazair Yahya
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mohd Azizi Abdul Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Khairil Anwar Abu Kassim
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Irfan Bahiuddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mohd Hatta Mohammed Ariff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohd Ibrahim Shapiai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zakaria, N.J., Shapiai, M.I., Fauzi, H. et al. Gradient-Based Edge Effects on Lane Marking Detection using a Deep Learning-Based Approach. Arab J Sci Eng 45, 10989–11006 (2020). https://doi.org/10.1007/s13369-020-04918-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-020-04918-4

Keywords

Search

Navigation