Triangulation of remote sensing, social sensing, and geospatial sensing for flood mapping, damage estimation, and vulnerability assessment

Flood events cause substantial damage to infrastructure and disrupt livelihoods. There is a need for the development of an innovative, open-access and real-time disaster map pipeline which is automatically initiated at the time of a flood event to highlight flooded regions, potential damage and vulnerable communities. This can help in directing resources appropriately during and after a disaster to reduce disaster risk. To implement this pipeline, we explored the integration of three heterogeneous data sources which include remote sensing data, social sensing data and geospatial sensing data to guide disaster relief and response. Remote sensing through satellite imagery is an effective method to identify flooded areas where we utilized existing deep learning models to develop a pipeline to process both optical and radar imagery. Whilst this can offer situational awareness right after a disaster, satellite-based flood extent maps lack important contextual information about the severity of structural damage or urgent needs of affected population. This is where the potential of social sensing through microblogging sites comes into play as it provides insights directly from eyewitnesses and affected people in real-time. Whilst social sensing data is advantageous, these streams are usually extremely noisy where there is a need to build disaster relevant taxonomies for both text and images. To develop a disaster taxonomy for social media texts, we conducted literature review to better understand stakeholder information needs. The final taxonomy consisted of 30 categories organized among three high-level classes. This built taxonomy was then used to label a large number of tweet texts (~ 10,000) to train machine learning classifiers so that only relevant social media texts are visualized on the disaster map. Moreover, a disaster object taxonomy for social media images was developed in collaboration with a certified emergency manager and trained volunteers from Montgomery County, MD Community Emergency Response Team. In total, 106 object categories were identified and organized as a hierarchical  taxonomy with  three high-level classes and 10 sub-classes. This built taxonomy will be used to label a large set of disaster images for object detection so that machine learning classifiers can be trained to effectively detect disaster relevant objects in social media imagery. The wide perspective provided by the satellite view combined with the ground-level perspective from locally collected textual and visual information helped us in identifying three types of signals: (i) confirmatory signals from both sources, which puts greater confidence that a specific region is flooded, (ii) complementary signals that provide different contextual information including needs and requests, disaster impact or damage reports and situational information, and (iii) novel signals when both data sources do not overlap and provide unique information. We plan to fuse the third component, geospatial sensing, to perform flood vulnerability analysis to allow easy identification of areas/zones that are most vulnerable to flooding. Thus, the fusion of remote sensing, social sensing and geospatial sensing for rapid flood mapping can be a powerful tool for crisis responders.