\section{Motivation}

Climate change results in an increased probability of extreme weather events causing, among others, severe floods \cite{hallegatte2013future}, landslides \cite{kainthura2022hybrid}, and glacier collapses \cite{rounce2023global}. Such natural hazards can cause significant casualties, put critical infrastructures at risk of failure, and decrease the quality of human lives.
Near real-time mapping of natural hazards is emerging priority for the support of natural disaster relief, risk management, and informing governmental policy decisions.

Advances in deep learning (DL) have resulted in a set of tools for earth monitoring \cite{zhu2017deep,wang2022self}. For instance, DL models have shown their advantages for land cover classification tasks \cite{wang2020weakly} and segmentation of floods \cite{fraccaro2022deploying}.
However, these approaches are designed for one specific task in one geographic region based on specific frequency bands of satellite data. 
Therefore, DL models used to map specific natural hazards struggle with their generalization to other types of natural hazards in unseen regions (see \cite{lunga2021resflow}). 
Therefore, adapting DL models to, for example, unseen natural hazards requires additional data labeled. Data annotation takes a significant amount of time, prolongs model fine-tuning, and generates additional costs.

In this work, we propose a methodology to significantly improve generalizability of DL natural hazards mappers based on pre-training on a suitable pre-task. Our approach supports the development of foundation models for earth monitoring, such as \cite{sun2022ringmo}, with the objective of directly segmenting unseen natural hazards across unseen geographic regions.
Our contributions are as follows: First, we demonstrate across four U-Net architectures that our approach significantly improves the generalizability of DL models for the segmentation of unseen natural hazards. Second, we show that our approach generalizes over novel geographic regions and different frequency bands of satellite data without access to data from the target domain. 
Third, by leveraging characteristics of unlabeled images from the target domain that are publicly available, our approach is able to further improve the generalization behavior without fine-tuning.

\section{Methodology}

In contrast to previous generations of geospatial DL models that are, if at all, pre-trained on the natural image domain (e.g., MS-COCO), we propose to pre-train models on a more suitable pre-task. 
In particular, we select the localization of buildings in the xBD dataset \cite{gupta2019xbd} as a suitable pre-task due to the dataset size and geographic diversity allowing us for an extensive pre-training on a simple localization task.

For training, we follow the settings proposed in the xView2 challenge \cite{durnov2020xview}: We utilize the well-established U-Net architecture with ResNet-34 \cite{he2016deep}, SeNet-154 \cite{hu2018squeeze}, SeResNeXt-50 \cite{hu2018squeeze,xie2017aggregated}, DPN-92 \cite{chen2017dual} as backbones and a weighted combination of focal and dice loss. All models are pre-trained over 50 epochs with early stopping by means of AdamW optimizer with learning rate of 0.00015. We apply a threshold to each mask produced by our models to classify pixels as \textit{background} or \textit{natural hazard}. This threshold is determined by the 95\% percentile of the pixel distribution per image.
    
After pre-training, we evaluate the generalization properties of the models on three downstream tasks. In particular, we evaluate the performance of our models on the segmentation of flood events from the Sen1Floods11 dataset \cite{bonafilia2020sen1floods11} (composed of 430 satellite images by Sentinel-1 capturing eleven globally-distributed flood events), the segmentation of fractures in glaciers from the Antarctic fracture detection dataset \cite{lai2020vulnerability} (consisting of 38 satellite images of six antarctic glaciers with fractures), and the segmentation of landslides from the HR-GLDD dataset \cite{meena2022hr} (containing 1,758 landslide instances from ten globally-distributed regions). These datasets consist of satellite images of unseen natural hazards in novel geographic regions and comprise different frequency bands. This setting allows us to holistically evaluate the generalization of the models.

For benchmarking, we compare the performance of our models with two different random baselines (i.e., uniform noise and masks generated by randomly initialized U-Net models) and the well-established segmentation model DeepLabV3 \cite{chen2017rethinking}. 
We evaluate the generalizability in two settings: without any domain adaptation and when unsupervised domain adaptation was applied.  
Therefore, for unsupervised domain adaptation, we make use of the available unlabeled data to update the statistics of the batch normalization layers of our models and thereby adapt our models to the domain of the downstream tasks. We evaluate different scenarios as part of $k$-shot adaptation (i.e., including $k \in \{1, 5, 10, 50\}$ unlabeled images from the target domain).
For evaluation, we calculate balanced accuracy, IoU, and F1 Score. Since the two classes of natural hazard and background are highly imbalanced but equally important for achieving the task, we only report balanced accuracy.

\section{Evaluation of Results}
In Figure \ref{fig:eval-results}, we present our results. First, we find that the performance of DeepLabV3 pre-trained on the MS-COCO dataset does not significantly vary from random performance for the segmentation of natural hazards. Unsurprisingly, the model predicts each pixel to belong to the background class as none of the classes encountered in pre-training are part of the target domain. Thus, DeeplabV3 does not generalize to the segmentation of natural hazards and requires fine-tuning.
Second, without access to any data from the target domain and across different unseen geographic regions, we confirm that performance of all four models is significantly better than the baselines for the segmentation of landslide and fractures in glaciers performing one-tailed Wilcoxon Rank tests with 0.01 significance level. For the segmentation of floods, two out of four models are significantly better than the baselines. 
Third, by utilizing unlabeled images as part of the unsupervised domain adaptation, we observe a general trend. For the majority of the models, an increase in the number of unlabeled images for unsupervised domain adaptation leads to improvement of the model performance. 
We only observe individual performance drops (e.g., zero-shot adaptation vs. 5-shot adaptation of SeNet-154 in Figure \ref{fig:antarctic}). This can be explained by non-optimal thresholding based on changing mask distributions. 


\begin{figure}[ht]
     \centering
        \begin{subfigure}[b]{0.33\textwidth}
        \tiny
        \begin{tabular}{l S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]}
        \toprule
              {\# images} & {ResNet-34} & {SeNet-154} & {SeResNeXt-50} & {DPN-92} \\
              \midrule
              0 & 0.5278 & 0.5120 & 0.5468*** & 0.6147*** \\
              \midrule
              1 & 0.5359 & 0.5089 &	0.5483 & 0.6197 \\
              5 & 0.5677 & 0.5147 &	0.5529 & 0.6345 \\
              \bottomrule
             \end{tabular}
         \centering
         \includegraphics[width=1\textwidth]{figures/sen1flood11_bal_acc_perc_thr.pdf}
         \caption{Sen1Floods11}
         \label{fig:sen1floods11}
     \end{subfigure}
     \vspace{0.5cm}
     \begin{subfigure}[b]{0.33\textwidth}
        \tiny
        \begin{tabular}{l S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]}
        \toprule
              {\# images} & {ResNet-34} & {SeNet-154} & {SeResNeXt-50} & {DPN-92} \\
              \midrule
              0 & 0.6443*** &	0.7366*** & 0.6641*** & 0.6244*** \\
              \midrule
              1 & 0.6573 &	0.7119 & 0.6603 & 0.6220 \\
              5 & 0.6605 &	0.6162 & 0.6748 & 0.6183 \\
              \bottomrule
             \end{tabular}
           \centering
         \includegraphics[width=1\textwidth]{figures/glaciers_bal_acc_perc_thr.pdf}
         \caption{Antarctic Glaciers}
         \label{fig:antarctic}
     \end{subfigure}
     \vspace{0.5cm}
     \begin{subfigure}[b]{0.33\textwidth}
         \tiny
        \begin{tabular}{l S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]S[table-format=1.2]}
        \toprule
              {\# images} & {ResNet-34} & {SeNet-154} & {SeResNeXt-50} & {DPN-92} \\
              \midrule
              0 & 0.6110*** & 0.6024*** & 0.5633*** & 0.5523*** \\
              \midrule
              1 & 0.6147 & 0.6021 &	0.5736 & 0.5508 \\
              5 & 0.6177 & 0.5896 & 0.5554 & 0.5396 \\
              \bottomrule
             \end{tabular}
         \centering
         \includegraphics[width=1\textwidth]{figures/landslide_bal_acc_perc_thr.pdf}
         \caption{Landslide}
         \label{fig:landslide}
     \end{subfigure}
     \caption{Generalizability of four U-Net models pre-trained on building localization in satellite images on three segmentation downstream tasks: (a) flood mapping, (b) fractures in glaciers, and (c) landslide mapping. Graphs depict mean $\pm$ std over 5 seeds, *** stands for statistically significant performance.}
     \label{fig:eval-results}
\end{figure}

For the Sen1Floods11 dataset, we observe that without access to any information from the target domain (i.e., zero-shot adaptation), the DPN-92 model achieves a balanced accuracy score of 61.47\%, as shown in Figure \ref{fig:sen1floods11}. When adapting the model to the target domain, the performance increases up to 65.72\% given 50 unlabeled images from the target domain. 
For the segmentation of fractures in glaciers shown in Figure \ref{fig:antarctic}, the ResNet-34 model achieves a score of 64.43\% in the zero-shot adaptation, This score is increased up to 71.42\% with 50-shot adaptation. 
Finally, for the segmentation of landslide, the ResNet-34 model achieves an initial performance of 61.10\% that is improved up to 62.63\% with 50 unlabeled images from the target domain as depicted in Figure \ref{fig:landslide}.

\section{Conclusion}

In this work, we demonstrate that a suitable pre-training improves the generalization of DL models for segmentation of unseen natural hazards.
Importantly, our approach is not limited to a specific frequency band or geographical region. Our results show that unsupervised domain adaptation can improve the performance of all considered models based on very few unlabeled samples.

\bibliographystyle{IEEEbib}
