Comparison of block-based stereo and semi-global algorithm and effects of pre-processing and imaging parameters on tree disparity map
Introduction
Determining tree canopy characterization of orchards is a non-destructive precision activity, which implicates measuring and obtaining exact knowledge of the geometry and structure of the trees for orchard management. There is a whole range of agricultural operations including the fertilization, pesticide treatments, crop training and irrigation which depend almost on the structural and geometric properties of trees crown (Rosell and Sanz, 2012; Usha and Singh, 2013).
Researches have been continued to a variety of non-destructive techniques for the measurement of tree canopy structural characteristics such as volume, foliage and leaf area index. It can be achieved by different detection approaches. The use of laser sensors (Naesset, 1997a, b; Aschoff et al., 2004; Van der Zande et al., 2006; Rosell et al., 2009a, b), as well as digital photographs (Phattaralerphong and Sinoquet, 2005; Leblanc et al., 2005), light sensors (Giuliani et al., 2000), high-resolution radar images (Bongers, 2001), stereo images (Andersen et al., 2005; Rovira-Mas et al., 2005; Kise and Zhang, 2006; Müller-Linow et al., 2015), high-resolution X-ray computed tomography (Stuppy et al., 2003; Milien et al., 2012) or ultrasonic sensors (Giles et al., 1988; Zaman and Salyani, 2004; Zaman and Schumann, 2005; Solanelles et al., 2006) offer the innovative solutions for the problem of structural assessment in trees (Rosell and Sanz, 2012). For example, Giles et al. (1987; 1988, 1989a, b) discussed the use of ultrasonic sensors to measure canopy volume in peach and apple trees. They developed this technique to improve the process of pesticide application. The measurement system was based on three ultrasonic sensors mounted at different heights of an air-blast orchard sprayer. The results showed the pesticide saving up to 52% in apples.
Despite recent efforts, little research has been done on 3D modeling of a tree by machine vision. In particular, the vision-based measurement methods are the nondestructive and effective way to determine external plant features (Yeh et al., 2014; Moriondo et al., 2016). Stereo vision is a method for the extraction of 3D information from digital images in machine vision. In this method, images are captured by the stereo camera, and disparity map is computed for each stereo pair to obtain 3D data. An important pre-step in correctly matching the stereo images and in precisely computing the depth in the stereo vision system is calibration. It is the process of estimating intrinsic and extrinsic parameters of the camera and needed for rectification and un-distortion of images. In order to represent the correct amount of disparity, the cameras used to capture the component images should be sufficiently aligned and/or rectified to determine a plurality of depth planes (a depth map). Also, the lighting is important because it affects camera and imaging parameters such as ISO speed, exposure time, metering mode, etc. (Shah et al., 2017; Jafari Malekabadi et al., 2017, 2018).
A large number of algorithms for stereo correspondence and disparity map have been developed (Scharstein and Szeliski, 2002), but the relative evaluation and comparative study of such techniques are limited (Mroz and Breckon, 2012). There are reasons why such a comparison of algorithms is valuable. Obviously, it allows us: 1- to motivate us to develop better algorithms, 2- to analyze algorithm characteristics and to improve overall implementation by focusing on subcomponents, 3- to ensure that algorithm performance is not sensitive to the setting of magical parameters. 4- It provides a design for special applications (Szeliski and Zabih, 2000).
Some algorithms and task related conditions were compared in several studies. In a study, the systematic procedure was developed to find the parameters that best sense the desired field of view. They distinguished the best combination of baseline length and suitable focal length for an agricultural robot. Nevertheless, late researches have tended to pick the best combination and have not examined the relationship between baseline adjustment and the accuracy of the 3D projection from the cameras (Rovira-Mas et al., 2010). In the other study, the effects of different stereo camera baselines on the accuracy of 3D projections generated from disparity maps were investigated. The results showed a correlation between stereo camera baseline and valid surface areas of the target object. This finding can be useful for researchers wanting to design and develop an effective stereo camera system and improve the quality of its 3D projection (Boonsuk, 2016). Gaujoux et al. (2015) determined the best technical conditions for intraoperative photography. Either smartphone camera, a bridge camera, or a single-lens reflex camera was used and photographs were taken under various standard conditions. The results showed that flash should be avoided and scialytic low-powered light should be used without focus. ISO speed should be as low as possible, shutter speed should be above 1/100 s, and aperture should be as narrow as possible, above f/8.
Researchers compared different algorithmic elements and they studied testing and ranking methodology where different algorithms. Generally, the development of stereo vision algorithms highly stimulated by this methodology and it focused on attaining well performance on a partly engineered image test set of static scenes. This test set is different from the imagery occurrence in the deployment of stereo vision systems in the automotive environment (Scharstein and Szeliski, 2002). Dall’Asta and Roncella (2014) compared some stereo matching algorithms (local and global) which were very popular in computer vision. The results showed the completeness of the digital surface models was usually good and error maps analysis had allowed explaining their quality. Least Square Semi-Global Matching algorithms were still the most accurate approach, but it presented noisy data in low-contrast or blurred regions where semi-global matching provided better results. Vidas (2016) selected a set of 15 stereo algorithms, mostly with real-time performance, which were categorized and evaluated on several NIR image datasets including single stereo pair and stream datasets. This comparison indicated that adaptive support weight and belief propagation algorithms had the highest accuracy of all fast methods, but also longer run times (2–3 seconds). On the other hand, faster algorithms (that achieve 30 or more fps on a single thread) usually performed an order of magnitude worse when measuring the percentage of incorrectly computed pixels.
Although many studies have been done on comparison stereo matching algorithms and study of effected parameters on these algorithms, there is not universally published experimental evaluation for stereo image pairs matching of the tree. Information of disparity maps can be used to predict yield, fertilizer application in citrus crops, water consumption or biomass. Specially, the results of this research can be used for spraying trees by variable rate sprayers. For this purpose, after analyzing the disparity maps, distance from tree to sprayer, the range of tree cover (canopy), the density of the tree cover and canopy volume will be determined in next. The block-based stereo and semi-global algorithms with 5 implementations were compared to calculate tree disparity maps. Also, the effects of ISO speed, exposure time, metering mode (imaging and camera parameters) and image Un-distorting and rectifying (pre-processing) on the disparity map of tree pictures were investigated.
Section snippets
Stereo vision system and image acquisition
To calculate tree volume (immersion method and mathematical formula) and verify the results of the stereo vision system, an artifact tree (cherry) was made with dimensions of 50 × 70 cm in conical shape. The structure of the tree and its components were similar to the real tree but in smaller scale. The artifact tree was positioned in a room which equipped with the controlled lighting system to investigate imaging parameters. The pair of images were acquired using two cameras (Canon A800) for
Results and discussion
After camera calibration and parameters calculation, images were un-distorted and rectified. Calibration error obtained 0.8 Pixel that was suitable to implement a stereo vision system. Images 1, 2 and 3 in Fig. 1 show original stereo images, un-distorted images and rectified images, respectively. The implementation of algorithms was performed on the rectified images and the disparity maps were calculated for images as shown in Fig. 2, Fig. 3.
In general, the algorithm based semi-global matching
Conclusions
The block-based stereo and semi-global algorithms with 5 implementations were compared to calculate tree disparity maps. Also, the effects of ISO speed, exposure time, metering mode (imaging parameters) and image un-distortion and rectification (pre-processing) on disparity map of tree pictures were evaluated. From the obtained results, it can be concluded that:
- •
The semi-global algorithm was better than block-based stereo algorithm.
- •
Un-rectified images had error in values of disparity map.
- •
Imaging
Acknowledgments
The authors gratefully acknowledge the financial support provided by Ferdowsi University of Mashhad (Grant No. 31500).
References (56)
- et al.
Geometric plant properties by relaxed stereo vision using simulated annealing
Comput. Electron. Agric.
(2005) - et al.
Sprayer control by sensing orchard crop characteristics: orchard architecture and spray liquid savings
J. Agric. Eng. Res.
(1989) - et al.
A comparative evaluation of combined feature detectors and descriptors in different color spaces for stereo image matching of tree
Sci. Hortic.
(2018) - et al.
Methodology comparison for canopy structure parameters extraction from digital hemispherical photography in boreal forest
Agric. Forest Meteorol.
(2005) - et al.
Visualization of the 3D structure of the graft union of grapevine using X-ray tomography
Sci. Hortic.
(2012) - et al.
Use of digital images to disclose canopy architecture in olive tree
Sci. Hortic.
(2016) Estimating timber volume of forest stands using airborne laser scanner data
Remote Sens. Environ.
(1997)Determination of mean tree height of forest stands using airborne laser scanner data
ISPRS J. Photogramm. Remote Sens.
(1997)- et al.
A review of methods and applications of the geometric characterization of tree crops in agricultural activities
Comput. Electron. Agric.
(2012) - et al.
Obtaining the three-dimensional structure of tree orchards from remote 2D terrestrial LIDAR scanning
Agric. For. Meteorol.
(2009)
Creation of three-dimensional crop maps based on aerial stereo images
Biosyst. Eng.
Design parameters for adjusting the visual field of binocular stereo cameras
Biosyst. Eng.
An electronic control system for pesticide application proportional to the canopy width of tree crops
Biosyst. Eng.
Three-dimensional analysis of plant structure using high-resolution X-ray computed tomography
Trends Plant Sci.
Potential applications of remote sensing in horticulture-a review
Sci. Hortic.
Influence of measurement set-up of ground-based LIDAR for derivation of tree structure
Agric. For.Meteorol.
An automated growth measurement system for leafy vegetables
Biosyst. Eng.
Describing forest stands using terrestrial laserscanning
Conference Proceedings ISPRS Conference. ISPRS International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. XXXV, Part B
Pmbp: patchmatch belief propagation for correspondence field estimation
Int. J. Comput. Vis.
Semi-global matching in object space
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.
Methods to assess tropical rain forest canopy structure: an overview
Plant Ecol.
Investigating effects stereo camera baseline accuracy 3d projection industrial robotic applications
Int. J. Eng. Res. Innov.
Camera Calibration Toolbox for Matlab
Learning OpenCV: Computer Vision with the OpenCV Libraryw
A comparison of semiglobal and local dense matching algorithms for surface reconstruction
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.
Sparse stereo disparity map densification using hierarchical image segmentation
13th International Symposium, ISMM 2017
Multispectral imaging in reflectance and photo-induced luminescence modes: a user manual
The British Museum, Charisma Project
Comparison of Open Source Stereo Vision Algorithms. MSc Degree
Cited by (6)
Stereo matching algorithm based on deep learning: A survey
2022, Journal of King Saud University - Computer and Information SciencesResearch on Strawberry Positioning Technology including Maturity Classification
2022, ACM International Conference Proceeding SeriesRoadside pedestrian detection and location based on binocular machine vision and RetinaNet
2021, Journal of Image and GraphicsLane Offset Survey for One-Lane Horizontal Curvatures Using Binocular Stereo Vision Measurement System
2021, Journal of Surveying EngineeringStereo matching algorithm based on hybrid convolutional neural network and directional intensity difference
2021, International Journal of Emerging Technology and Advanced EngineeringConverged classification network for matching cost computation
2020, International Journal of Advanced Science and Technology