登录 EN

添加临时用户

基于扩散模型的异构三维形状注册方法研究

Research on Heterogeneous 3D Shape Registration via Diffusion Model

作者:郭宸
  • 学号
    2021******
  • 学位
    硕士
  • 电子邮箱
    101******com
  • 答辩日期
    2024.05.13
  • 导师
    黄儒麒
  • 学科名
    数据科学和信息技术
  • 页码
    63
  • 保密级别
    公开
  • 培养单位
    600 清华-伯克利深圳学院
  • 中文关键词
    三维形状注册;扩散模型;多视角渲染;图像形变;三维高斯喷洒
  • 英文关键词
    3D shape registration; Diffusion models; multi-view rendering; image morphing; 3D Gaussian Splatting

摘要

本文提出一个新颖的基于图像扩散模型的异构三维形状注册框架。传统的几何注册方法假设源形状与目标形状之间的形变满足某种几何正则化条件,无法准确建模异构形状的大幅度形变关系。尽管近年来有一系列工作通过人工标记对应点或者特定类别上的数据先验解决前叙困难,但这些方法需要复杂的人工输入或者特定类别上的异构三维数据集作为基础,难以应对一般的异构三维形状注册问题。本文算法的核心思想是将三维形状通过渲染变成多视角二维图像,利用在大规模图像数据上训练的扩散模型获得在不同视角下图像的形变过程。然后将形变图像序列通过动态重建获得源形状与目标形状之间具有语义信息的逐渐变化的三维形状序列。最后,源形状与目标形状间的大幅度形变被分解为形状序列间若干小幅度形变,我们采用传统的非刚性形状注册沿着中间形状序列逐一注册,最终获得源形状到目标形状的注册结果。本文在EBCM,SHREC07等具有挑战性的数据集上测试了提出的方法与若干基线方法,实验结果验证了本文提出方法不依赖手动标注关键点和三维先验,相较于同类方法在输出映射的光滑性与准确性上取得显著的优势,并且接近于当前最先进的基于关键点标注的方法。本文方法生成的插值形状不仅有助于异构形状注册,在形状生成和姿态转移等应用中也有一定的应用潜力。总的来说,本文展示了利用可扩展扩散模型捕获的图像级语义知识来推进三维形状分析和操作任务的潜力。

This thesis proposes a novel heterogeneous 3D shape registration framework based on image diffusion models. Traditional geometric registration methods assume that the deformation between the source shape and the target shape satisfies some geometric regularization condition, which cannot accurately model the significant deformation of heterogeneous shapes. Although there has been a series of works in recent years that have solved the mentioned difficulty by manually labeling landmark points or learning data priors on specific categories, these methods require complex manual input or heterogeneous 3D datasets on specific categories as the basis, making it difficult to deal with general heterogeneous 3D shape registration problems. The core idea of the algorithm in this thesis is to render 3D shapes into multi-view 2D images, and use a diffusion model trained on large-scale image data to obtain the deformation process of images from different perspectives. Then, the deformation image interpolation sequence is dynamically reconstructed to obtain a gradually changing 3D shape sequence with semantic information between the source shape and the target shape. Finally, the significant deformation between the source shape and the target shape is decomposed into several small deformations between the intermediate shape sequences. We use traditional non-rigid shape registration to register along the intermediate shape sequence one by one, and finally obtain the registration results from the source shape to the target shape. This thesis tested the proposed method and several baseline methods on challenging datasets such as EBCM and SHREC07. The experimental results verified that the proposed method does not rely on manual keypoint annotation and 3D priors. Compared with similar methods, it has significant advantages in the smoothness and accuracy of output mapping, and is close to the current state-of-the-art landmark annotation based methods. The interpolation shapes generated by the method in this thesis not only contribute to the registration of heterogeneous shapes, but also have certain potential applications in shape generation and pose transfer. Overall, this thesis demonstrates the potential of utilizing image level semantic knowledge captured by scalable diffusion models to advance 3D shape analysis and manipulation tasks.