AI classification of marine birds and mammals based on aerial imagery of the German North and Baltic Seas

Following the current expansion of offshore constructions for the production of renewable energy as well as shipping traffic, assessments of impacts on marine ecosystems are becoming increasingly important. Thus, accurate knowledge of the spatial and temporal distribution of animal species is mandatory regarding the preservation of biodiversity and management of offshore wind farms and further economic activities. High-resolution optical imagery of airborne remote sensing sensors enables the observation of marine birds and mammals within large ocean areas. However, the identification of features at the ocean surface as well as the separation of animals and further objects, such as wave structures, ships or buoys, requires time-consuming visual inspection of the acquired image sequences by trained personnel. Here, we apply an AI-based approach to automatically detect and classify various features above the sea surface based on aerial imagery of the German North Sea and Baltic Sea. A large number of optical images at a spatial resolution of 2 cm have been acquired by the German Federal Agency for Nature Conservation (BfN) during repeated monitoring flights since 2018. These images are preprocessed and geolocated by assigning respective auxillary informations to create an extensive database on marine animal observations. The AI method which we are developing has to be responsible both for detecting birds in images, and for tracking instances of a same element present on multiple frames in order to avoid counting an individual multiple times. Some of the main challenges which will have to be dealt with are the following. First, luminosity conditions cannot be controled and might be suboptimal in a large fraction of the images, rendering animals completely white or black, or difficult to distinguish from the background. Second, smaller animals might consist only of little pixel blobs, and thus be difficult to distinguish. Third, flying birds might have shadows, which, while bird-shaped, must not be classified as birds. Fourth, in bird flocks overlapping tricks the AI into detecting one bird instead of several ones, which renders tracking significantly more challenging. We aim at tackling the third and fourth issues by incorporating cinematic estimation of the plane‘s and animal‘s movements, and estimating the direction of the sun in each frame, into the tracking system. In the future, our system will be used by the German Federal Agency for Nature Conservation (BfN) to monitor bird and mammal populations, and evaluate the effectiveness of preservation measures.