Evaluation of the effectiveness of two approaches to building damage detection with satellite imagery

Abstract

This study addresses the approaches for satellite image analysis to assess infrastructure damage. The main aim is to conduct a comprehensive comparative analysis of the effectiveness of two key machine learning approaches: specialized semantic segmentation based on the U-Net architecture and generalized visual analysis using large vision-language models. The object of the research is the process of quantitatively benchmarking these two distinct approaches to determine their practical applicability for multi-class damage classification.

The research material is the publicly available xView2 dataset. The methods involved two parallel experiments. For the semantic segmentation approach, a U-Net model with an EfficientNet-B4 encoder was implemented and trained on 6-channel input data ("before" and "after" images) using a combined Dice and Focal loss function. For the vision-language models approach, the open-source LLaVA-1.5-7B model was evaluated in a zero-shot mode using advanced prompt engineering for an aggregative counting task. To enable a direct comparison, the standard Jaccard index was calculated based on the aggregated object counts for each damage class.

The results of the experiments revealed a significant performance disparity. The specialized U-Net model demonstrated high effectiveness, achieving an intersection over union score of 0.6141 on the test set. In contrast, the LLaVA model proved unsuitable for accurate quantitative analysis, yielding an extremely low Jaccard index of approximately 0.063, primarily due to its systemic failure to correctly identify and count objects (Recall ≈ 0.07). The scientific novelty lies in being the first study to quantitatively document this order-of-magnitude capability gap, confirming that for tasks requiring high-precision mapping, specialized segmentation models remain the indispensable tool.