Developing software for mapping urban radiative properties through analysis of street view imagery

The accuracy of urban weather and climate simulations is constrained by the availability of localized and detailed urban morphology data. In this evolving field, street view imagery is emerging as a complementary data source, as it becomes openly available through municipal data portals and through crowdsourced platforms like Kartaview and Mapillary. While advanced image analysis techniques exist and are widely used in adjacent fields, their application in weather and climate models remains limited. A key question is whether features extracted from these images can reach sufficient quality for use in urban modelling studies.

Within the Urban-M4 project, we are developing software to interactively explore street view imagery with a focus on use in microscale simulations with WRF and PALM. Our current interest is in estimating radiative properties of urban surfaces, in particular albedo and emissivity of roofs, roads, and facades.

So far, we are able to segment images and extract individual instances of buildings, roads, and other relevant features such as windows and doors. This is done using a combination of prompt-based object detection and labelling with the GroundingDINO model and instance segmentation with the Segment Anything Model (SAM). For the extracted instances, we estimate albedo by aggregating the brightness values of the annotated pixels. As a next step, we aim to add a material classification layer, which would allow for indirect estimation of both albedo and emissivity based on known radiation properties of different materials.

All functionality is bundled in a Python package called streetscapes. It includes tools for retrieving and storing collections of street view images from various sources in a structured way. Images and their derived data (e.g. segmentations, masks, statistics) follow a consistent naming convention, so that processed outputs can be automatically linked to their source. In addition, a metadata system is maintained that keeps track of image properties such as location, file identifiers, and any extracted statistics. This setup allows users to query and filter the dataset for specific use cases. The package also includes functionality to apply image segmentation and other analysis methods, and to prepare outputs (e.g. raster files) suitable for use in urban climate models. The code leaves room for possible extensions such as linking images to individual buildings or applying photogrammetry for advanced texture mapping.

The code is openly available at: https://github.com/Urban-M4/streetscapes.