Applications of Computer Vision in Underwater Ecology: A Case Study from the Northeast Pacific

The world's oceans are undergoing rapid changes due to climate change and other anthropogenic impacts, affecting marine species' distribution, abundance, and behavior. Traditional ecological monitoring methods struggle to keep pace with these transformations, especially in underwater habitats. Recently, computer vision techniques have emerged to enhance the efficiency of video and image-based underwater monitoring. These methods facilitate the detection and classification of objects in visual data, potentially streamlining the process of counting and classifying marine organisms. However, the adoption of computer vision in marine ecology has been slow, partly due to its inaccessibility to ecologists and the lack of easily adaptable tools for ecological monitoring.

This study investigates the application and validation of computer vision techniques for monitoring underwater pelagic macrofaunal diversity, using a case study from coastal British Columbia. Over 9000 hours of underwater video were collected from four sites over 18 months, using mounted cameras programmed to record five minutes of video every hour. 

Due to the infrequency of organisms present in the videos and challenges with underwater visibility, we created a stepwise iterative screening process to sequentially refine video data and aid the image annotation process. This involved using unsupervised classifiers (e.g. ResNet-18) to assist in reducing the number of background (i.e. 'empty water') images shown to annotators. To address the scarcity of annotations for certain taxa, an ‘adjacency filter’ was employed to increase the number of annotated frames for rare species. A rigorous QAQC process ensured standardization and minimized inter-annotator bias. Finally, a supervised computer vision model (YOLOv8) was trained with approximately 240,000 annotations to assess the presence and abundance of marine species over 18 months in the area. This approach provided high-resolution temporal data on the diversity and abundance of pelagic fish and gelatinous zooplankton at these sites.

Here, we detail the process of employing these computer vision techniques for long-term underwater ecological monitoring, emphasizing accessibility for ecologists. A stepwise method for adapting computer vision techniques to achieve biodiversity monitoring objectives is presented, highlighting the strengths and limitations of our approach. We address our work in the context of the key barriers facing computer vision in underwater ecology, and provide tools for researchers seeking to incorporate AI in image or video-based marine research. Finally, we propose future directions for integrating these technologies into new and existing monitoring programs, and suggest priority areas for future research to advance the use of computer vision in underwater ecological monitoring.