EGU24-2934, updated on 08 Mar 2024
EGU General Assembly 2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Accelerating Marine UAV Drone Image Analysis with Sliced Detection and Clustering (MBARI SDCAT)

Duane R. Edgington, Danelle E. Cline, Thomas O'Reilly, Steven H.D. Haddock, John Phillip Ryan, Bryan Touryan-Schaefer, William J. Kirkwood, Paul R. McGill, and Rob S. McEwen
Duane R. Edgington et al.
  • Monterey Bay Aquarium Research Institute (MBARI), Moss Landing, CA, USA (

Uncrewed Aerial Vehicles (UAVs) can be a cost-effective solution for capturing a comprehensive view of surface ocean phenomena to study marine population dynamics and ecology. UAVs have several advantages, such as quick deployment from shore, low operational costs, and the ability to be equipped with various sensors, including visual imaging systems and thermal imaging sensors. However, analyzing high-resolution images captured from UAVs can be challenging and time-consuming, especially when identifying small objects or anomalies. Therefore, we developed a method to quickly identify a diverse range of targets in UAV images.

We will discuss our workflow for accelerating the analysis of high-resolution visual images captured from a Trinity F90+ Vertical Take-Off and Landing (VTOL) drone in near-shore habitats around the Monterey Bay region in California at approximately 60 meters altitude. Our approach uses a state-of-the-art self-distillation with knowledge (DINO) transformer foundation model and multi-scale, sliced object detection (SAHI) methods to locate a wide range of objects, from small to large, such as schools or individual jellyfish, flocks of birds, kelp forests or kelp fragments, small debris, occasional cetaceans, and pinnipeds. To make the data analysis more efficient, we create clusters of similar objects based on visual similarity, which can be quickly examined through a web-based interface. This approach eliminates the need for previously labeled objects to train a model, optimizing limited human resources. Our work demonstrates the useful application of state-of-the-art techniques to assist in the rapid analysis of images and how this can be used to develop a recognition system based upon machine-learning for the rapid detection and classification of UAV images. All of our work is freely available as open-source code.

How to cite: Edgington, D. R., Cline, D. E., O'Reilly, T., Haddock, S. H. D., Ryan, J. P., Touryan-Schaefer, B., Kirkwood, W. J., McGill, P. R., and McEwen, R. S.: Accelerating Marine UAV Drone Image Analysis with Sliced Detection and Clustering (MBARI SDCAT), EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-2934,, 2024.

Supplementary materials

Supplementary material file

Comments on the supplementary material

AC: Author Comment | CC: Community Comment | Report abuse

supplementary materials version 1 – uploaded on 07 May 2024, no comments