Big Data Analysis of Antarctic Ice Structures and Subglacial Lakes: Utilizing Moving IQR for Radar Intensity Processing

The structure of Antarctic ice preserves the sequence of ice deposition, offering insights into ancient environmental conditions. Organisms discovered beneath the ice sheets, spanning from hundreds to thousands of meters in thickness, hold information on survival in extreme environments. Antarctic ice investigations are conducted using radar systems mounted on helicopters or vehicles, generating vast datasets covering hundreds of kilometers. Analyzing this large-scale data is essential to reduce time and cost for detecting ice structures and subglacial lakes. In this study, we developed algorithms for ice structure analysis and subglacial lake detection using big data analysis techniques, specifically outlier detection methods applied to radar signal values. Utilizing radar signal values represented in an 800x83,344 matrix, we employed the Spark platform with specifications of 400 cores and 1.6TB of memory for data analysis. To facilitate data processing in Spark, the data was transformed into a 3x66,675,200 dataframe after uploading to HDFS. Outlier detection, using the Moving Interquartile Range (IQR), identified abrupt changes in signal values based on columns, adjusting the IQR's range and scale to optimize the results. Detected outlier values were normalized within a 0-255 range and visualized based on intensity. Results revealed that using the Moving IQR for radar imagery processing effectively detected localized changes as the range increased; however, detection rates decreased with larger scales. Analyzing radar exploration results in a big data environment is anticipated to significantly reduce time and costs compared to traditional methods, contributing to Antarctic exploration and climate change response efforts.