ABSTRACT
For ecological protection of the ocean, biologists usually conduct line-transect vessel surveys to measure sea species’ population density within their habitat (such as dolphins). However, sea species observation via vessel surveys consumes a lot of manpower resources and is more challenging compared to observing common objects, due to the scarcity of the object in the wild, tiny-size of the objects, and similar-sized distracter objects (e.g., floating trash). To reduce the human experts’ workload and improve the observation accuracy, in this paper, we develop a practical system to detect Chinese White Dolphins in the wild automatically. First, we construct a dataset named Dolphin-14k with more than 2.6k dolphin instances. To improve the dataset annotation efficiency caused by the rarity of dolphins, we design an interactive dolphin box annotation strategy to annotate sparse dolphin instances in long videos efficiently. Second, we compare the performance and efficiency of three off-the-shelf object detection algorithms, including Faster-RCNN, FCOS, and YoloV5, on the Dolphin-14k dataset and pick YoloV5 as the detector, where a new category (Distracter) is added to the model training to reject the false positives. Finally, we incorporate the dolphin detector into a system prototype, which detects dolphins in video frames at 100.99 FPS per GPU with high accuracy (i.e., 90.95 [email protected]).
- Jay Barlow, Megan Ferguson, William Errin, Lisa Ballance, Tim Gerrodette, Gerald Joyce, Colin Macleod, Keith Mullin, Debi Palka, and Gordon Waring. 2005. Abundance and densities of beaked and bottlenose whales (family Ziphiidae). Journal of Cetacean Research and Management 7 (01 2005).Google Scholar
- S. Buckland, D. Anderson, K. Burnham, Jeffrey Laake, David Borchers, and Len Thomas. 2001. Introduction to Distance Sampling: Estimating Abundance of Biological Populations. Vol. xv.Google Scholar
- Joana Castro, Francisco O. Borges, André Cid, Marina I. Laborde, Rui Rosa, and Heidi C. Pearson. 2021. Assessing the Behavioural Responses of Small Cetaceans to Unmanned Aerial Vehicles. Remote Sensing 13, 1 (2021). https://doi.org/10.3390/rs13010156Google ScholarCross Ref
- Abhishek Dutta and Andrew Zisserman. 2019. The VIA Annotation Software for Images, Audio and Video. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM ’19). ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3343031.3350535Google ScholarDigital Library
- Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440–1448.Google ScholarDigital Library
- Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 580–587.Google ScholarDigital Library
- Emilio Guirado, Siham Tabik, Marga L. Rivas, Domingo Alcaraz-Segura, and Francisco Herrera. 2019. Whale counting in satellite and aerial images with deep learning. Scientific Reports 9, 1 (03 Oct 2019), 14259. https://doi.org/10.1038/s41598-019-50795-9Google ScholarCross Ref
- T. Jefferson. 2018. Hong Kong’s Indo-Pacific Humpback Dolphins (Sousa chinensis): Assessing Past and Future Anthropogenic Impacts and Working Toward Sustainability. Aquatic Mammals 44(2018), 711–728.Google ScholarCross Ref
- Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2117–2125.Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740–755.Google ScholarCross Ref
- Malte Pedersen, Joakim Bruslund Haurum, Rikke Gade, and Thomas B. Moeslund. 2019. Detection of Marine Animals in a New Underwater Dataset with Varying Visibility. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.Google Scholar
- Sarah Piwetz, Thomas A. Jefferson, and Bernd Würsig. 2021. Effects of Coastal Construction on Indo-Pacific Humpback Dolphin (Sousa chinensis) Behavior and Habitat-Use Off Hong Kong. Frontiers in Marine Science 8 (2021), 196. https://doi.org/10.3389/fmars.2021.572535Google ScholarCross Ref
- Débora Pollicelli, Mariano Coscarella, and Claudio Delrieux. 2020. RoI detection and segmentation algorithms for marine mammals photo-identification. Ecological Informatics 56 (2020), 101038. https://doi.org/10.1016/j.ecoinf.2019.101038Google ScholarCross Ref
- Joseph Redmon. 2013–2016. Darknet: Open Source Neural Networks in C. http://pjreddie.com/darknet/.Google Scholar
- Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 779–788.Google ScholarCross Ref
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497(2015).Google Scholar
- Vito Renò, Gianvito Losapio, Flavio Forenza, Tiziano Politi, Ettore Stella, Carmelo Fanizza, Karin Hartman, Roberto Carlucci, Giovanni Dimauro, and Rosalia Maglietta. 2020. Combined Color Semantics and Deep Learning for the Automatic Detection of Dolphin Dorsal Fins. Electronics 9, 5 (2020). https://doi.org/10.3390/electronics9050758Google ScholarCross Ref
- Heather R. Smith, Daniel P. Zitterbart, Thomas F. Norris, Michael Flau, Elizabeth L. Ferguson, Colin G. Jones, Olaf Boebel, and Valerie D. Moulton. 2020. A field comparison of marine mammal detections via visual, acoustic, and infrared (IR) imaging methods offshore Atlantic Canada. Marine Pollution Bulletin 154 (2020), 111026. https://doi.org/10.1016/j.marpolbul.2020.111026Google ScholarCross Ref
- Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. 2019. Fcos: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9627–9636.Google ScholarCross Ref
- Ursula K. Verfuss, Douglas Gillespie, Jonathan Gordon, Tiago A. Marques, Brianne Miller, Rachael Plunkett, James A. Theriault, Dominic J. Tollit, Daniel P. Zitterbart, Philippe Hubert, and Len Thomas. 2018. Comparing methods suitable for monitoring marine mammals in low visibility conditions during seismic surveys. Marine Pollution Bulletin 126 (2018), 1–18. https://doi.org/10.1016/j.marpolbul.2017.10.034Google ScholarCross Ref
- Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh. 2020. CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 390–391.Google ScholarCross Ref
- Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. 2019. Detectron2. https://github.com/facebookresearch/detectron2.Google Scholar
- Peiqin Zhuang, Linjie Xing, Yanlin Liu, Sheng Guo, and Yu Qiao. 2017. Marine Animal Detection and Recognition with Advanced Deep Learning Models. In Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, September 11-14, 2017(CEUR Workshop Proceedings, Vol. 1866), Linda Cappellato, Nicola Ferro, Lorraine Goeuriot, and Thomas Mandl (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-1866/paper_166.pdfGoogle Scholar
- Daniel P. Zitterbart, Heather R. Smith, Michael Flau, Sebastian Richter, Elke Burkhardt, Joe Beland, Louise Bennett, Alejandro Cammareri, Andrew Davis, Meike Holst, Caterina Lanfredi, Hanna Michel, Michael Noad, Kylie Owen, Aude Pacini, and Olaf Boebel. 2020. Scaling the Laws of Thermal Imaging–Based Whale Detection. Journal of Atmospheric and Oceanic Technology 37, 5(2020), 807 – 824. https://doi.org/10.1175/JTECH-D-19-0054.1Google ScholarCross Ref
Index Terms
- Chinese White Dolphin Detection in the Wild
Recommendations
Dolphin Detection and Tracking
WACVW '15: Proceedings of the 2015 IEEE Winter Applications and Computer Vision WorkshopsDetecting and tracking social marine mammals, including bottlenose dolphins, can help to explain their social dynamics, predict their behavior, and measure the impact of human interference. Multi-camera setups provide an opportunity to record the ...
Dynamics modeling and simulation for a gliding robotic dolphin
2016 IEEE International Conference on Robotics and Biomimetics (ROBIO)This paper provides a detailed hydrodynamic analysis of the gliding motion for a gliding robotic dolphin. In order to realize both high maneuverability and long endurance, a novel gliding robotic dolphin is developed through introducing a buoyancy-driven ...
Numerical Exploration on Pitching Motion of Robotic Dolphin Realized by Pectoral Fin
2021 IEEE International Conference on Mechatronics and Automation (ICMA)This paper proposes an elliptical trajectory pectoral fin flapping motion model for robotic dolphin used to obtain a better pitching performance. A fin-flapping dolphin model is established and fluid dynamic method is used for simulation of pitching ...
Comments