配准是数字病理领域重要的数据分析、数据融合工具。作为深入分析及整合同一组织跨切片信息的关键手段,配准算法有助于直观呈现不同染色下同一组织结构的形态变化、生物标志物表达情况。虽然现有深度学习配准算法的研究取得飞速发展,但该领域仍面临挑战:病理切片存在明显的初始方位错位时,变换参数计算错误导致配准精度较差;病理图像具备窄色域特、纹理重复特性,预训练模型数据迁移不当导致配准质量下降;病理图像标注数据成本较高。基于上述背景,本文提出并实现了一种创新的基于自监督学习的端到端病理图像仿射配准算法。相比现有技术,该算法的优势在于流程简单、运行速度快、鲁棒性强且减少了对人工标注的需求,从而利于扩大配准技术在临床诊断与研究中的应用规模与效能。 首先,本文提出并实现了一种端到端的弱监督仿射配准方案。运用深度学习技术,以一步式策略完成图像的预对齐和全局变换,极大地简化配准流程、缩短配准时间,免于多阶段式配准的组织分割、预对齐的多步参数调优步骤。本方案通过计算输入图像的旋转角度标签和图像相似度实现模型的训练和优化,能大幅减少图像标注的时间成本,有利于大规模配准数据集的建立、提升配准算法的应用价值。 在成功构建端到端仿射配准框架后,本文提出了一种针对病理图像无方向性的自监督任务。对在引入了自监督任务的端到端仿射配准算法的实验结果表明,自监督任务有效促进网络训练的稳定收敛,并增强对配准关键特征的捕获能力。通过一系列包括消融实验、与主流方法的对比实验以及不同染色模式数据配准的测试,本文展示了基于自监督任务的端到端仿射配准算法在处理存在大旋转初始角度错位的数据时,在配准精度、运行时间、鲁棒性方面的优势。此外,本文还将该算法拓展应用到非刚性配准任务中,采用更公正的方式与现有算法进行比较,初步揭示了该配准方法在临床多切片分析及半自动标注等场景的应用潜力。 综上所述,端到端配准作为一种具备广阔应用前景的配准范式,有望在多切片分析、三维重建、多模态信息融合等方面实现组织结构的便捷、快速对齐,助力组织病理学诊断效率的提升,为病理图像分析自动化提供基础。
Registration is an important tool for data analysis and fusion in the field of digital pathology. As a crucial means for in-depth analysis and integration of information across sections of the same tissue, registration algorithms facilitate the intuitive presentation of morphological changes and biomarker expression in the same tissue structure under different staining conditions. Despite the rapid development of research on deep learning-based registration algorithms, challenges remain in this field: the accuracy of registration is poor due to incorrect calculation of transformation parameters when there is a significant initial misalignment in pathology sections; the narrow color gamut and repetitive texture characteristics of pathology images lead to a decline in registration quality due to improper data transfer of pre-trained models; and the high cost of annotating pathology images. Based on the this background, this paper proposes and implements an innovative end-to-end pathological image affine registration algorithm based on self-supervised learning. Compared with existing technologies, this algorithm has the advantages of a simple process, fast operation speed, strong robustness, and reduced need for manual annotation, thereby facilitating the expansion of the application scale and effectiveness of registration technology in clinical diagnosis and research. Firstly, this paper proposes and implements an end-to-end weakly supervised affine registration framework. Utilizing a deep learning architecture, it completes the pre-alignment and global transformation of images in a one-step strategy, greatly simplifying the registration process and shortening the registration time, avoiding the multi-stage registration‘s tissue segmentation and multi-step parameter tuning of pre-alignment. This framework achieves the model‘s training and optimization by calculating the rotation angle labels of the input images and image similarity, which can significantly reduce the time cost of image annotation, facilitating the establishment of large-scale registration datasets and enhancing the application value of the registration algorithm. After successfully constructing an end-to-end affine registration framework, this paper introduces a self-supervised task tailored for the directionless nature of pathological images. The experimental results of the end-to-end affine registration algorithm, which incorporates the self-supervised task, show that the self-supervised task effectively promotes stable convergence of network training and enhances the ability to capture key registration features.Through a series of experiments, including ablation studies, comparisons with existing methods, and tests on data registration with different staining patterns, this paper demonstrates the advantages of the end-to-end affine registration algorithm based on self-supervised tasks in terms of registration accuracy, operation time, and robustness when dealing with data presenting significant initial rotational angle differences. Furthermore, this paper extends the application of this algorithm to non-rigid registration tasks, comparing it more fairly with existing algorithms, and preliminarily reveals the potential of this registration method in clinical multi-slice analysis and semi-automatic annotation scenarios. In summary, as a registration paradigm with broad application prospects, end-to-end registration is expected to facilitate and expedite the alignment of tissue structures in areas such as multi-slice analysis, three-dimensional reconstruction, and multimodal information fusion. This aims to enhance the efficiency of histopathological diagnosis and provide a foundation for the automation of pathological image analysis.