Skip to main content
SpringerLink
Log in

An on-board vision sensor system for small unmanned vehicle applications

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes an on-board vision sensor system that is developed specifically for small unmanned vehicle applications. For small vehicles, vision sensors have many advantages, including size, weight, and power consumption, over other sensors such as radar, sonar, and laser range finder, etc. A vision sensor is also uniquely suited for tasks such as target tracking and recognition that require visual information processing. However, it is difficult to meet the computing needs of real-time vision processing on a small robot. In this paper, we present the development of a field programmable gate array-based vision sensor and use a small ground vehicle to demonstrate that this vision sensor is able to detect and track features on a user-selected target from frame to frame and steer the small autonomous vehicle towards it. The sensor system utilizes hardware implementations of the rank transform for filtering, a Harris corner detector for feature detection, and a correlation algorithm for feature matching and tracking. With additional capabilities supported in software, the operational system communicates wirelessly with a base station, receiving commands, providing visual feedback to the user and allowing user input such as specifying targets to track. Since this vision sensor system uses reconfigurable hardware, other vision algorithms such as stereo vision and motion analysis can be implemented to reconfigure the system for other real-time vision applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Garcia, R., Valavanis, K., Kontitsis, M.: A multiplatform on-board processing system for miniature unmanned vehicles. In: Proceedings of the 2006 IEEE International Conference on Robotics and Automation, ICRA 2006, 15–19, pp. 2156–2163 (2006)

  2. “Power consumption and the modern geek”, 8 2010 [Online]. Available: http://www.extremetech.com/-article2/-0,2845,1937997,00.asp

  3. Naish P., Bishop P.: Designing ASICS. Halsted Press, New York (1988)

    Google Scholar 

  4. Anderson, J. D., Lee, D.-J., Edwards, B., Archibald, J., Greco, C. R.: Real-time feature tracking on an embedded vision sensor for small vision-guided unmanned vehicles. In: Computational Intelligence in Robotics and Automation, 2007. CIRA 2007. International Symposium on, 20–23, pp. 55–60 (2007)

  5. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. pp. 151–158. Springer (1994)

  6. Konolige, K.: Small vision system: Hardware and implementation. In: Eighth International Symposium on Robotics Research (1997)

  7. Rock, S., Frew, E., Jones, H., LeMaster, E., Woodley, B.: “Combined cdgps and vision-based control of a small autonomous helicopter. In: American Control Conference, 1998. Proceedings of the 1998, vol. 2, 21–26, pp. 694–698 (1998)

  8. Ettinger S., Nechyba M., Ifju P., Waszak M.: Vision-guided flight stability and control for micro air vehicles. Adv. Robot. 17(7), 617–640 (2003)

    Article  Google Scholar 

  9. Hrabar, S., Sukhatme, G.: Omnidirectional vision for an autonomous helicopter. In: Proceedings of the 2003 IEEE International Conference on Robotics and Automation. ICRA ’03 vol. 1, 14–19, pp. 558–563 (2003)

  10. Frew, E., McGee, T., Kim, Z., Xiao, X., Jackson, S., Morimoto, M., Rathinam, S., Padial, J., Sengupta, R.: Vision-based road-following using a small autonomous aircraft. In: Aerospace Conference, 2004. Proceedings. 2004 IEEE, vol. 5, 6–13, pp. 3006–3015 (2004)

  11. Cornall, T., Egan, G.: Measuring horizon angle from video on a small unmanned air vehicle. In: 2nd International Conference on Autonomous Robots and Agents. Citeseer, pp. 339–344 (2004)

  12. Dobrokhodov, V., Kaminer, I., Jones, K., Ghabcheloo, R.: Vision-based tracking and motion estimation for moving targets using small uavs. In: American Control Conference, 2006, 14–16 (2006)

  13. Proctor A., Johnson E., Apker T.: Vision-only control and guidance for aircraft. J. Field Robot. 23(10), 863–890 (2006)

    Article  MATH  Google Scholar 

  14. MacLean, W.: An evaluation of the suitability of fpgas for embedded vision systems. In: Computer Vision and Pattern Recognition—Workshops, 2005. IEEE Computer Society Conference on CVPR Workshops, 25–25, pp. 131–131 (2005)

  15. Cardon, D., Fife, W., Archibald, J., Lee, D.: Fast 3d reconstruction for small autonomous robots. In: Industrial Electronics Society, 2005. IECON 2005. 31st Annual Conference of IEEE, 6–10 (2005)

  16. Torres-Huitzil, C., Arias-Estrada, M.: An fpga architecture for high speed edge and corner detection. In: CAMP ’00: Proceedings of the Fifth IEEE International Workshop on Computer Architectures for Machine Perception (CAMP’00), p. 112. IEEE Computer Society, Washington, D.C. (2000)

  17. Darabiha, A., Rose, J., MacLean, W.: Video-rate stereo depth measurement on programmable hardware In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 203–210 (2003)

  18. Se S., Jasiobedzki P.: Stereo-vision based 3d modeling and localization for unmanned vehicles. Int. J. Intell. Control Syst. 13(1), 47–58 (2008)

    Google Scholar 

  19. Aubépart, F., Franceschini, N.: Bio-inspired optic flow sensors based on fpga: Application to micro-air-vehicles. Microprocess. Microsyst. 31(6), 408–419 (2007), special Issue on Sensor Systems. [Online]. Available: http://www.sciencedirect.com/-science/-article/-B6V0X-4N4YMM2-2/-2/-58a7ddaeb91047ed991d37c199583481

  20. Song, T.L., Lee, D.G.: A probabilistic nearest neighbor filter algorithm for m validated measurements. In: IEEE Transactions on Signal Processing, vol. 54, no. 7, pp. 2797–2802, July 2006

  21. Tissainayagam P., Suter D.: Assessing the performance of corner detectors for point feature tracking applications. Image Vis. Comput. 22(8), 663–679 (2004)

    Article  Google Scholar 

  22. Tommasini, T., Fusiello, A., Trucco, E., Roberto, V.: Making good features track better. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Citeseer, pp. 178–183 (1998)

  23. Nickels, K., Hutchinson, S.: Measurement error estimation for feature tracking. In: Proceedings of the International Conference on Robotics and Automation, vol. 4, pp. 3230–3235 (1999)

  24. Fife W.S., Archibald J.K.: Reconfigurable on-board vision processing for small autonomous vehicles. EURASIP J. Embed. Syst. 2007(1), 33–33 (2007)

    Article  Google Scholar 

  25. Harris, C., Stephens, M.: A combined corner and edge detector. In: 4th Alvey Vision Conference, vol. 15, p. 50. Manchester (1988)

  26. Zabih, R., Woodfill, J.: Non-parametric Local Transforms for Computing Visual Correspondence. pp. 151–158. ECCV (2004)

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, Brigham Young University, Provo, UT, 84602, USA

    Beau J. Tippetts, Dah-Jye Lee & James K. Archibald

Authors
  1. Beau J. Tippetts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dah-Jye Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James K. Archibald
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dah-Jye Lee.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tippetts, B.J., Lee, DJ. & Archibald, J.K. An on-board vision sensor system for small unmanned vehicle applications. Machine Vision and Applications 23, 403–415 (2012). https://doi.org/10.1007/s00138-012-0413-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-012-0413-9

Keywords

Search

Navigation