Detection of Melt Ponds on Arctic Sea Ice from Infrared Images using AutoSAM

The Arctic is significantly affected by climate change, as evidenced by the constant decline of sea ice since the beginning of satellite observations. One driver of this transformation are melt ponds - pools of water that form as a result of melting sea ice during summer. Due to their darker color, they increase the absorption of incoming sunlight and accelerate ice melt. Accurate determination of melt pond extent and characteristics is considered a main factor in reducing uncertainty in Arctic climate models and sea ice concentration retrievals, but precise large scale observations are not available. Most knowledge to date is based on in-situ measurements, which are restricted to small areas. Satellite retrievals offer Arctic-wide coverage on a regular basis but lack resolution. To validate satellite measurements and allow observation at a moderate scale, helicopter-borne images are used. This ongoing work exploits a new dataset of helicopter-borne thermal infrared (TIR) imagery for melt pond retrieval. The derivation of geophysical parameters requires effective segmentation of different surface classes, which is challenged by temporally and spatially varying surface temperatures. We adapt and fine-tune AutoSAM, a prompt-free Segment Anything (SAM)-based segmentation tool that was introduced by Xinrong Hu et al. for medical imagery (Hu, X., Xu, X., & Shi, Y. (2023). How to Efficiently Adapt Large Segmentation Model (SAM) to Medical Images. arXiv preprint arXiv:2306.13731). Initial results with a limited number of annotated images indicate promising outcomes in the generalization of AutoSAM to cases that are rare in the training set, compared to U-Net and PSP-Net approaches. Beyond the scope of this project, this could serve as an example of how to use SAM as a segmentation tool for the remote sensing domain, which is typically hampered by the lack of labeled training data. Code and first results are provided at https://github.com/marlens123/autoSAM_pond_segmentation.