Elsevier

Scientia Horticulturae

Volume 247, 15 March 2019, Pages 264-274
Scientia Horticulturae

Comparison of block-based stereo and semi-global algorithm and effects of pre-processing and imaging parameters on tree disparity map

https://doi.org/10.1016/j.scienta.2018.12.033Get rights and content

Highlights

  • The semi-global and block-based stereo algorithms were compared to calculate tree disparity maps.

  • The effects of ISO speed, exposure time, metering mode, image un-distortion and rectification were evaluated.

  • Un-rectified images had error in values of disparity map.

  • Imaging parameters were affected on light intensity and in result on disparity map.

  • Best disparity map obtained in conditions that ISO speed and exposure time had fewer value and metering mode was set to pattern mode.

Abstract

One method of creating a 3D model of a tree is stereo vision technique. Tree geometric characteristics have been used to predict yield, fertilizer application in citrus crops, water consumption or biomass. So, a disparity map and 3D model of the tree can be useful. There are different algorithms to match the stereo pair for calculating disparity maps. Many parameters affect the disparity map, such as the lighting that affects camera and imaging parameters, the calibration that needed to rectify the image, etc. In this study, the block-based stereo and semi-global algorithms with 5 implementations were compared to calculate tree disparity maps. Also, the effects of ISO speed, exposure time, metering mode (imaging parameters) and image un-distortion and rectification (pre-processing) were investigated. The results showed that the semi-global algorithm was better than the block-based stereo algorithm. Windows size affected on disparity maps and best results obtained when this parameter was set to default. Maximum disparity value limits search space. Best results obtained in real value. For smaller value than real value, disparity map was not calculated outside that maximum disparity. Therefore, disparities or pixels missed. It observed that image rectifying, ISO speed, exposure time and metering mode were affected on disparity map. When pictures did not rectify, disparity map calculated but with some errors. Camera parameters were affected on light intensity of pictures. Best disparity map obtained in conditions that ISO speed and exposure time had fewer value and metering mode was set to pattern mode and these parameters were similar for both left and right images.

Introduction

Determining tree canopy characterization of orchards is a non-destructive precision activity, which implicates measuring and obtaining exact knowledge of the geometry and structure of the trees for orchard management. There is a whole range of agricultural operations including the fertilization, pesticide treatments, crop training and irrigation which depend almost on the structural and geometric properties of trees crown (Rosell and Sanz, 2012; Usha and Singh, 2013).

Researches have been continued to a variety of non-destructive techniques for the measurement of tree canopy structural characteristics such as volume, foliage and leaf area index. It can be achieved by different detection approaches. The use of laser sensors (Naesset, 1997a, b; Aschoff et al., 2004; Van der Zande et al., 2006; Rosell et al., 2009a, b), as well as digital photographs (Phattaralerphong and Sinoquet, 2005; Leblanc et al., 2005), light sensors (Giuliani et al., 2000), high-resolution radar images (Bongers, 2001), stereo images (Andersen et al., 2005; Rovira-Mas et al., 2005; Kise and Zhang, 2006; Müller-Linow et al., 2015), high-resolution X-ray computed tomography (Stuppy et al., 2003; Milien et al., 2012) or ultrasonic sensors (Giles et al., 1988; Zaman and Salyani, 2004; Zaman and Schumann, 2005; Solanelles et al., 2006) offer the innovative solutions for the problem of structural assessment in trees (Rosell and Sanz, 2012). For example, Giles et al. (1987; 1988, 1989a, b) discussed the use of ultrasonic sensors to measure canopy volume in peach and apple trees. They developed this technique to improve the process of pesticide application. The measurement system was based on three ultrasonic sensors mounted at different heights of an air-blast orchard sprayer. The results showed the pesticide saving up to 52% in apples.

Despite recent efforts, little research has been done on 3D modeling of a tree by machine vision. In particular, the vision-based measurement methods are the nondestructive and effective way to determine external plant features (Yeh et al., 2014; Moriondo et al., 2016). Stereo vision is a method for the extraction of 3D information from digital images in machine vision. In this method, images are captured by the stereo camera, and disparity map is computed for each stereo pair to obtain 3D data. An important pre-step in correctly matching the stereo images and in precisely computing the depth in the stereo vision system is calibration. It is the process of estimating intrinsic and extrinsic parameters of the camera and needed for rectification and un-distortion of images. In order to represent the correct amount of disparity, the cameras used to capture the component images should be sufficiently aligned and/or rectified to determine a plurality of depth planes (a depth map). Also, the lighting is important because it affects camera and imaging parameters such as ISO speed, exposure time, metering mode, etc. (Shah et al., 2017; Jafari Malekabadi et al., 2017, 2018).

A large number of algorithms for stereo correspondence and disparity map have been developed (Scharstein and Szeliski, 2002), but the relative evaluation and comparative study of such techniques are limited (Mroz and Breckon, 2012). There are reasons why such a comparison of algorithms is valuable. Obviously, it allows us: 1- to motivate us to develop better algorithms, 2- to analyze algorithm characteristics and to improve overall implementation by focusing on subcomponents, 3- to ensure that algorithm performance is not sensitive to the setting of magical parameters. 4- It provides a design for special applications (Szeliski and Zabih, 2000).

Some algorithms and task related conditions were compared in several studies. In a study, the systematic procedure was developed to find the parameters that best sense the desired field of view. They distinguished the best combination of baseline length and suitable focal length for an agricultural robot. Nevertheless, late researches have tended to pick the best combination and have not examined the relationship between baseline adjustment and the accuracy of the 3D projection from the cameras (Rovira-Mas et al., 2010). In the other study, the effects of different stereo camera baselines on the accuracy of 3D projections generated from disparity maps were investigated. The results showed a correlation between stereo camera baseline and valid surface areas of the target object. This finding can be useful for researchers wanting to design and develop an effective stereo camera system and improve the quality of its 3D projection (Boonsuk, 2016). Gaujoux et al. (2015) determined the best technical conditions for intraoperative photography. Either smartphone camera, a bridge camera, or a single-lens reflex camera was used and photographs were taken under various standard conditions. The results showed that flash should be avoided and scialytic low-powered light should be used without focus. ISO speed should be as low as possible, shutter speed should be above 1/100 s, and aperture should be as narrow as possible, above f/8.

Researchers compared different algorithmic elements and they studied testing and ranking methodology where different algorithms. Generally, the development of stereo vision algorithms highly stimulated by this methodology and it focused on attaining well performance on a partly engineered image test set of static scenes. This test set is different from the imagery occurrence in the deployment of stereo vision systems in the automotive environment (Scharstein and Szeliski, 2002). Dall’Asta and Roncella (2014) compared some stereo matching algorithms (local and global) which were very popular in computer vision. The results showed the completeness of the digital surface models was usually good and error maps analysis had allowed explaining their quality. Least Square Semi-Global Matching algorithms were still the most accurate approach, but it presented noisy data in low-contrast or blurred regions where semi-global matching provided better results. Vidas (2016) selected a set of 15 stereo algorithms, mostly with real-time performance, which were categorized and evaluated on several NIR image datasets including single stereo pair and stream datasets. This comparison indicated that adaptive support weight and belief propagation algorithms had the highest accuracy of all fast methods, but also longer run times (2–3 seconds). On the other hand, faster algorithms (that achieve 30 or more fps on a single thread) usually performed an order of magnitude worse when measuring the percentage of incorrectly computed pixels.

Although many studies have been done on comparison stereo matching algorithms and study of effected parameters on these algorithms, there is not universally published experimental evaluation for stereo image pairs matching of the tree. Information of disparity maps can be used to predict yield, fertilizer application in citrus crops, water consumption or biomass. Specially, the results of this research can be used for spraying trees by variable rate sprayers. For this purpose, after analyzing the disparity maps, distance from tree to sprayer, the range of tree cover (canopy), the density of the tree cover and canopy volume will be determined in next. The block-based stereo and semi-global algorithms with 5 implementations were compared to calculate tree disparity maps. Also, the effects of ISO speed, exposure time, metering mode (imaging and camera parameters) and image Un-distorting and rectifying (pre-processing) on the disparity map of tree pictures were investigated.

Section snippets

Stereo vision system and image acquisition

To calculate tree volume (immersion method and mathematical formula) and verify the results of the stereo vision system, an artifact tree (cherry) was made with dimensions of 50 × 70 cm in conical shape. The structure of the tree and its components were similar to the real tree but in smaller scale. The artifact tree was positioned in a room which equipped with the controlled lighting system to investigate imaging parameters. The pair of images were acquired using two cameras (Canon A800) for

Results and discussion

After camera calibration and parameters calculation, images were un-distorted and rectified. Calibration error obtained 0.8 Pixel that was suitable to implement a stereo vision system. Images 1, 2 and 3 in Fig. 1 show original stereo images, un-distorted images and rectified images, respectively. The implementation of algorithms was performed on the rectified images and the disparity maps were calculated for images as shown in Fig. 2, Fig. 3.

In general, the algorithm based semi-global matching

Conclusions

The block-based stereo and semi-global algorithms with 5 implementations were compared to calculate tree disparity maps. Also, the effects of ISO speed, exposure time, metering mode (imaging parameters) and image un-distortion and rectification (pre-processing) on disparity map of tree pictures were evaluated. From the obtained results, it can be concluded that:

  • The semi-global algorithm was better than block-based stereo algorithm.

  • Un-rectified images had error in values of disparity map.

  • Imaging

Acknowledgments

The authors gratefully acknowledge the financial support provided by Ferdowsi University of Mashhad (Grant No. 31500).

References (56)

  • F. Rovira-Mas et al.

    Creation of three-dimensional crop maps based on aerial stereo images

    Biosyst. Eng.

    (2005)
  • F. Rovira-Mas et al.

    Design parameters for adjusting the visual field of binocular stereo cameras

    Biosyst. Eng.

    (2010)
  • F. Solanelles et al.

    An electronic control system for pesticide application proportional to the canopy width of tree crops

    Biosyst. Eng.

    (2006)
  • W. Stuppy et al.

    Three-dimensional analysis of plant structure using high-resolution X-ray computed tomography

    Trends Plant Sci.

    (2003)
  • K. Usha et al.

    Potential applications of remote sensing in horticulture-a review

    Sci. Hortic.

    (2013)
  • D. Van der Zande et al.

    Influence of measurement set-up of ground-based LIDAR for derivation of tree structure

    Agric. For.Meteorol.

    (2006)
  • Y.-H.F. Yeh et al.

    An automated growth measurement system for leafy vegetables

    Biosyst. Eng.

    (2014)
  • T. Aschoff et al.

    Describing forest stands using terrestrial laserscanning

    Conference Proceedings ISPRS Conference. ISPRS International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. XXXV, Part B

    (2004)
  • F. Besse et al.

    Pmbp: patchmatch belief propagation for correspondence field estimation

    Int. J. Comput. Vis.

    (2014)
  • F. Bethmann et al.

    Semi-global matching in object space

    Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

    (2015)
  • F. Bongers

    Methods to assess tropical rain forest canopy structure: an overview

    Plant Ecol.

    (2001)
  • W. Boonsuk

    Investigating effects stereo camera baseline accuracy 3d projection industrial robotic applications

    Int. J. Eng. Res. Innov.

    (2016)
  • J.Y. Bouguet

    Camera Calibration Toolbox for Matlab

    (2004)
  • G. Bradski et al.

    Learning OpenCV: Computer Vision with the OpenCV Libraryw

    (2008)
  • E. Dall’Asta et al.

    A comparison of semiglobal and local dense matching algorithms for surface reconstruction

    Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

    (2014)
  • S. Drouyer et al.

    Sparse stereo disparity map densification using hierarchical image segmentation

    13th International Symposium, ISMM 2017

    (2017)
  • J. Dyer et al.

    Multispectral imaging in reflectance and photo-induced luminescence modes: a user manual

    The British Museum, Charisma Project

    (2013)
  • C. Emmanouil

    Comparison of Open Source Stereo Vision Algorithms. MSc Degree

    (2015)
  • Cited by (6)

    View full text