Estimating urban albedo and emissivity from street view imagery

Weather and climate simulations continue to evolve towards higher resolutions. This allows them to resolve small-scale processes more explicitly, but the added value is constrained by the availability of accurate localized data, particularly in urban areas where there is a large variety in urban structures and surface properties. Currently, mesoscale models like WRF rely on typological classifications such as the Local Climate Zones. Despite their proven effectiveness, they bundle multiple properties into a single urban class, which means individual parameters cannot be represented independently. Recent studies have introduced fine-scale explicit datasets on various urban properties such as building heights and vegetation fraction. But to the best of our knowledge, local datasets of albedo and emissivity of urban surfaces are not available at scale. 

In the “Urban-M4” project, we are exploring whether street view imagery can provide these missing radiative properties for use in urban weather models. Such imagery is widely available nowadays, either as proprietary data (e.g. Google Streetview), but also increasingly as open data from municipalities or through crowdsourcing platforms such as Mapillary and Kartaview. Simultaneously, computer vision methods have become much more powerful. State of the art models can now perform advanced tasks including detection of a wide range of objects and materials based on free prompts. This allows us to extract individual buildings or building parts from street view images and analyse their characteristics. As a proxy for albedo, we have been experimenting with various brightness metrics of building pixels, resulting in a first preliminary map of façade albedo for Amsterdam based on 100k images. We are currently setting up an observational campaign to validate and refine this method. To eventually estimate emissivity as well, we are investigating the capability of existing computer vision models to recognize (urban) materials. 

We are developing this openly on GitHub, and to facilitate adoption the functionality is bundled in a Python package called ‘streetscapes’. It includes tools for retrieving images from various sources and running a number of computer vision models. While it is possible to automatically segments millions of images, the quality of the results is still affected by the heterogeneity of images and the varying accuracy of the models. Therefore, we aim to further develop the package to accommodate a ‘human-in-the-loop’ workflow, so it becomes manageable to inspect images and their metadata in a spatial context, and filter or modify images and metadata from a graphical interface. We have modified WRF to enable ingestion of 2D maps of urban albedo and emissivity and are preparing the first tests.