Enhancing Building Height Estimation through Occlusion Reduction with Advanced Deep Learning Models

Building heights play a crucial role in various urban research fields, including 3D modeling, urban environmental analysis, sustainable development, and urban planning and management. Numerous methods have been developed to derive building heights from different data sources, including street view imagery, which offers detailed, ground-level perspectives of buildings. However, occlusions from street elements such as trees and vehicles present significant challenges, especially in densely built or complex urban areas. To address this challenge, we propose the use of advanced deep learning models for occlusion reduction, enhancing building height estimation from street view images. As trees typically cause the most occlusion, we employ an open-set detector and a large segmentation deep neural network to create tree masks in the images. Subsequently, we use a stable diffusion model for image inpainting, restoring parts of buildings occluded by trees. These inpainted images are then processed through building instance segmentation, yielding clearer building boundaries for height estimation. Moreover, we integrate a single-view metrology-based height estimation method with a building footprint auxiliary approach, leveraging their respective strengths and mitigating the impact of varying distances between street view cameras and buildings. Our methodology is validated using a dataset comprising 954 buildings and 3814 images. Experimental results demonstrate that our approach increases the percentage of height estimates within a two-meter error margin by approximately 7%, confirming its effectiveness. This work offers a cost-effective solution for large-scale building height mapping and updating, and it opens new avenues for urban research requiring accurate building height data.