Waterline and obstacle detection in images from low-cost autonomous boats for environmental monitoring

Highlights

• A pixel-wise segmentation is used to separate water and non-water regions.

• The water/non-water mask is analyzed to infer the position of the waterline.

• Waterline information is used to reduce the search space for detecting obstacles.

Abstract

Waterline detection from images taken by cameras mounted on low-cost autonomous surface vehicles (ASVs) is a key process for obtaining a fast obstacle detection. Achieving an accurate waterline prediction is difficult due to the instability of the ASV on which the camera is mounted and the presence of reflections, illumination changes, and waves. In this work, we present a method for waterline and obstacle detection designed for low-cost ASVs employed in environmental monitoring. The proposed approach is made of two steps: (1) a pixel-wise segmentation of the current image is used to generate a binary mask separating water and non-water regions, (2) the mask is analyzed to infer the position of the waterline, which in turn is used for detecting obstacles. Experiments were carried out on two publicly available datasets containing floating obstacles such as buoys, sailing and motor boats, and swans moving near the ASV. Quantitative results show the effectiveness of the proposed approach with 98.8% pixel-wise segmentation accuracy running at 10 frames per second on an embedded GPU board.

Introduction

The water quality monitoring process costs every year more than 1 billion EUR at the European Union level [1]. In particular, analyses of pollution in large rivers or lakes supplying drinking water are in the range of 150,000 to 400,000 EUR. The current approach, based on field sample and laboratory analysis, is unable to assess temporal and spatial variation in the contaminants of concern. This means that, in case of accident, there is the possibility of taking inadequate and late decisions for mitigating the pollution impacts.

The use of autonomous surface vehicles (ASVs) for persistent large-scale monitoring of aquatic environments is a valid and efficient alternative to the traditional manual sampling approach [2]. ASVs are capable of undertaking long-endurance missions and carrying multiple sensors (e.g., for measuring electrical conductivity and dissolved oxygen) to obtain water quality indicators [3]. There exist commercial ASVs specifically developed for water quality monitoring. An example is the Lutra mono hull boat produced by Platypus,¹ shown in Fig. 1. Lutra boats are used in the EU-funded Horizon 2020 project INTCATCH,² which aims to develop a new paradigm in the monitoring of river and lake water quality, by bringing together, validating, and exploiting a range of innovative tools into a single efficient and user-friendly model.

To achieve true autonomous navigation, an ASV must sense its environment and avoid possible collisions. In this paper, we propose to use vision-based sensing to reach this goal, focusing on the domain of small, low-cost ASVs where sensors commonly utilized for localization purposes (e.g., LiDARs) and powerful processing units cannot be used due to price caps.

The contribution of this work is two-fold: (i) we present a pixel-wise deep learning based method able to segment images captured by cameras on low-cost ASV into water/non-water regions, and (ii) we describe an algorithm for extracting the horizon line (called waterline in this context) from the segmented images and for detecting potential floating obstacles. It is worth noticing that, the relationship between waterline and obstacles is heavily dependent on the pitch and roll angles of the ASV, which are only roughly measured by cheap gyroscope and accelerometer sensors available in low-cost boats. Our goal is to use the waterline to reduce the search space for the detection of potential obstacles and to deal with the pitch and roll movements, which can cause misdetections, by dynamically tracking over-time the obstacle positions. In particular, we use Convolutional Neural Networks (CNNs) for image segmentation, edge detection and RANSAC for waterline extraction, and a multi-object tracker based on cost computation.

Although CNNs were previously introduced in other mobile robotics domains (e.g., indoor wheeled and quadrotors), and ASV obstacle detection techniques were employed with classical computer vision methods (e.g., optical flow), the two approaches have never been combined yet.

We provide the complete source code of our method (see Section 6) and the dataset of images and videos created for testing (see the INTCATCH Vision Dataset.³ ) This dataset contains annotated data that can be used for developing supervised approaches for object detection.

The remainder of the paper is structured as follows. Anoverview of the INTCATCH system is given in Section 2 and related work is discussed in Section 3. The proposed segmentation algorithm is presented in Section 4, while the waterline and obstacle detection process is described in Section 5. Experimental results are shown in Section 6. Finally, conclusions are drawn in Section 7.