Interactive evaluation of street view image segmentations for urban parameter extraction - an illustration for Amsterdam

Outdoor heat exposure of citizens in urban areas becomes more critical to understand due to the compounding of urbanization and climate change. Outdoor heat load depends on the urban morphology, e.g. the reflectivity of the walls. However, spatial datasets of wall albedo values in cities are so far lacking, but street view images may act as potential sources to estimate albedo and develop such gridded datasets.

In the “Urban-M4” eScience project we explored how street view imagery can be used to derive radiative properties for use in urban weather models. This data is widely available, not only from proprietary sources (Google’s Streetview) but also from crowd-sourcing platforms such as Mapillary and Kartaview. With modern computer vision models, these images can be “segmented”; partitioning an image into district groups based on properties such as material, objects or other concepts. These segmentation techniques provide new opportunities to extract urban parameters from (street view) imagery, but the quality of these segmentations has yet to be established. We present a new tool to quickly evaluate segmentations of street view imagery.

With the ‘streetscapes’ Python package project users can download street view images and segment them, as well as export aggregated data for further analysis. However, it was difficult for end users to review the segmentation results and images within the Python interface. Therefore, we designed and built the “streetscapes explorer”, a browser based visual interface for analyzing the street view images as well as the segmentation results. Users can view the images on a map and filter these for information such as image source, tags, or segmentation labels. This allows users to quickly and easily assess the results produced by computer vision models. If after curation, the user is satisfied with the results, these can be exported to a format compatible with GIS programs for further analysis or processing such as rasterization.