Multimodal deep learning for entity relation extraction and spatiotemporal decision knowledge graph construction in earthquake emergency rescue

This paper proposes a novel framework integrating multimodal deep learning with spatiotemporal knowledge representation to enhance information processing and decision support in earthquake emergency rescue operations. We address the challenges of heterogeneous data integration through a Cross-modal Attention Fusion Network (CAFN) that dynamically aligns semantic information across textual, visual, and spatiotemporal modalities. A Transformer-based joint entity-relation extraction model simultaneously identifies disaster-related entities and their relationships with 89.0% F1 score, outperforming conventional approaches by 8.7%. We further develop a spatiotemporal knowledge graph representation with specialized reasoning mechanisms that explicitly model the dynamic nature of emergency scenarios. The resulting decision support system demonstrates a 94.0% F1 score in resource allocation and route planning tasks, exceeding traditional rule-based systems by 23.7%. Experimental results confirm the effectiveness of our approach in processing complex multimodal information and supporting time-critical decisions during earthquake emergency rescue operations.