滑坡、崩塌等地质灾害给人民生命财产安全和社会经济发展带来极大危害，因而对地质灾害的风险进行预测，可以为灾害防治提供切实有效的帮助。以往的地灾风险预测通常采用基于统计或者机器学习的方法进行分析，难以深度挖掘特征信息。近年来Transformer架构在自然语言处理等领域愈发受到关注，本身具有灵活性、并行性以及强大的表征能力，如果将其应用到地灾风险预测研究中，可以挖掘地灾数据的特征和点之间的内在关联信息。因此，基于“贵州省地质灾害防治指挥平台”项目，本文选择使用Transformer架构对地质灾害风险预测进行研究。构建适用于地灾风险预测的Transformer模型，并从分层架构与预训练模式两个方向加以改进，提升模型的地灾风险预测能力。论文的主要工作如下：（1）针对单地理点不同地理特征的融合问题，建立了以Transformer为基础架构的地灾风险预测模型Geological Disaster Transformer (GDT)。通过对原始地理数据采样精度调整、重采样与特征计算，构建地灾风险数据集。然后在Transformer基础上加入对高度、坡度、坡向等14维特征的编码模块，调整位置编码、归一化层等，通过自注意力机制实现对不同地理特征的深度交叉融合，从而得到地灾风险预测GDT模型。实际数据测试表明该模型预测能力较好。（2）针对多地理点多地理特征的融合问题，引入分层架构建立了地灾风险预测模型Hierarchical Geological Disaster Transformer (HGDT)。为了进一步考虑多点之间的关联性，依据分层思想在GDT基础上添加一层针对不同地理点的Transformer，改进为分层模型HGDT，分别从特征层面与点层面实现信息交互与融合，并对不同特征层面中的位置编码模块添加与否问题进行了研究。数据测试结果表明HGDT进一步提升了模型的地灾风险预测能力。（3）针对带标签地灾风险数据规模较小的问题，对GDT模型使用预训练与微调模式，引入更多无标注数据。风险标签标注需要一定成本，大量无标注数据没有使用可能导致信息损失。因此采用预训练和微调的模式，使用GDT对所有无标注地理数据进行重构预训练，让模型学习基础地理信息，进而在带标签地灾风险数据集上对模型进一步微调，从而获得了更好的地灾风险预测结果。
Landslides, collapses, and other geological disasters pose a significant threat to public safety, property, and socio-economic development, making the prediction of geological disaster risks crucial for the prevention and mitigation efforts. Traditional geological disaster risk predictions often used statistical or machine learning methods, which may struggle to deeply mine feature information. In recent years, the Transformer architecture, increasingly noticed in fields such as natural language processing, has demonstrated flexibility, parallelism, and powerful representation ability. If applied to geological disaster risk prediction research, it might uncover the inherent correlations between features and points in geological disaster data. Therefore, this thesis conducts research on geological disaster risk prediction based on the Transformer architecture, within the framework of the "Geological Disaster Prevention and Command Platform of Guizhou Province" project. We constructed a Transformer model tailored for geological disaster risk predictions and improved it from two aspects, hierarchical architecture and pre-training modes, to enhance the model’s predictive capabilities. The main contributions of the thesis are as follows:(1) To address the issue of integrating different geographical features at a single geographical point, we established the Geological Disaster Transformer (GDT) model based on the Transformer architecture. By adjusting the sampling precision of the original geographic data, resampling, and calculating features, we constructed a geological disaster risk dataset. Then we incorporated a module encoding 14-dimensional features such as elevation, slope, and aspect into the Transformer base, adjusting the position encoding and normalization layers, among other aspects. We achieved deep cross-integration of different geographic features through the self-attention mechanism, resulting in the geological disaster risk prediction GDT model. Actual data tests show that this model has good predictive performance.(2) To solve the problem of information fusion of multiple geographical points and features, Hierarchical Geological Disaster Transformer (HGDT), a model based on a hierarchical architecture, was introduced. For further consideration of the correlation between multiple points, a layer aimed at different geographical points’ Transformer was added on the basis of GDT, creating the hierarchical model HGDT. This model implements interaction and fusion of information from both the feature and point levels, and the study also addressed whether to include the position encoding module at different feature levels. Data testing results indicate that HGDT further enhances the predictive capability of the model for geological disaster risk.(3) Regarding the issue of the small scale of labeled geological disaster risk data, we utilized pre-training and fine-tuning modes with the GDT model, introducing more unlabeled data. Labeling risk requires certain costs, and unused large volumes of unlabeled data can lead to information loss. Therefore, we adopted a pre-training and fine-tuning model, where GDT first performed reconstruction pre-training on all unlabeled geographic data, allowing the model to learn basic geographic information. The model was then further fine-tuned on labeled geological disaster risk datasets, resulting in improved geological disaster risk predictions.
