登录 EN

添加临时用户

基于RGBD数据的人脸三维重建及其整形医学应用

3D Face Reconstruction Based on RGBD Data and Its Application in Plastic Surgery

作者:郝进
  • 学号
    2020******
  • 学位
    硕士
  • 电子邮箱
    hao******com
  • 答辩日期
    2023.05.19
  • 导师
    徐枫
  • 学科名
    软件工程
  • 页码
    57
  • 保密级别
    公开
  • 培养单位
    410 软件学院
  • 中文关键词
    人脸三维重建, RGBD数据, 多阶段融合方法, 纹理重建, 整形医学
  • 英文关键词
    3D face reconstruction, RGBD data, Multi-stage fusion method, Texture reconstruction, Plastic surgery

摘要

人脸三维重建是利用图像、视频等观测信息在数字世界重建三维人脸几何表面及纹理的技术,人脸三维重建已经被广泛应用于各个领域,例如人脸识别、整形医疗、虚拟数字人和影视动画等,在日常生活和工作生产中具有重要的应用价值。在面部整形医疗中,人脸三维重建可以帮助进行医学诊断、术前术后的效果评估。然而,目前的医用人脸三维重建依赖于三维扫描仪或CT等专业设备,要求患者在院内经医生指导进行数据采集。而随着深度传感器的普及,越来越多的便携式设备具备了采集三维信息的能力,为实现居家人脸三维重建及远程诊断和评估提供了可能性。 本文利用iPhone手机采集的RGBD数据,对人脸三维重建算法进行了深入研究,提出了一套快速、高精度的人脸三维几何重建和纹理重建方法。同时本文还开发了一套集数据采集、人脸三维重建以及在线可视化评估的辅助诊断系统。本文的工作探索了多帧RGBD数据的融合重建方法,证明了基于三维重建的面部整形远程医疗的可能性。本文的主要贡献包括: 1. 设计并实现了一种基于RGBD序列的人脸几何重建方法。该方法首先通过帧检测和提取,支持使用患者自行采集的非规范数据,避免了对患者的严格的数据采集和标注要求,其次提出了一种多阶段融合算法,针对iPhone获得的具有噪声的深度数据,逐步从粗糙到精细地进行数据融合从而实现完整三维人脸重建。本文实现了较为准确的三维人脸重建,保留了患者的病理特征。 2. 设计并实现了一种基于RGBD序列的人脸纹理重建方法。该方法首先通过帧选择策略,避免了所选帧中存在的人脸图像模糊问题,其次在所选帧上应用去光照技术,解决了极端光照下纹理重建难题,最后该方法通过拉普拉斯金字塔进行纹理融合,实现了高精度人脸纹理重建。 3. 设计并实现了一套集数据采集、人脸三维重建以及在线可视化评估的辅助诊断系统,该系统支持针对患者的语音与画面辅助的拍摄指导、数据上传、云端计算和网页端结果展示等辅助诊断全流程操作,实现完整的远程三维人脸辅助诊断与评估。

3D face reconstruction is a technique of reconstructing 3D face geometric surface and texture in the digital world using image, video and other observation information. 3D face reconstruction has been widely used in various fields, such as face recognition, plastic surgery medical treatment, virtual digital human and film animation, etc., and has important application value in daily life and work production. In facial plastic surgery medical treatment, 3D face reconstruction can help medical diagnosis, preoperative and postoperative effect evaluation. However, the current medical 3D face reconstruction relies on professional equipment such as 3D scanners or CT, which requires patients to be guided by doctors in the hospital for data acquisition. With the popularity of depth sensors, more and more portable devices have become capable of acquiring 3D information, offering the possibility of 3D reconstruction of the face at home and remote diagnosis and evaluation. In this thesis, RGBD data collected by iPhone is used to conduct an in-depth study of face 3D reconstruction algorithm, and a set of fast and high-precision face 3D geometric reconstruction and texture reconstruction methods are proposed. This thesis also develops an auxiliary diagnosis system integrating data acquisition, 3D face reconstruction and online visualization and evaluation. This work explores the fusion reconstruction method for multi-frame RGBD data and demonstrates the feasibility of remote medical care for facial plastic surgery based on 3D reconstruction. The main contributions of this thesis include: 1. A face geometry reconstruction method based on RGBD sequences is designed and implemented. This method first supports non-standard data collected by patients themselves through frame detection and extraction, avoiding strict data collection and annotation requirements, and second proposes a multi-stage fusion algorithm to gradually fuse the data from coarse to fine for the noisy depth data obtained by iPhone to achieve a complete 3D face reconstruction. This thesis provides relatively accurate 3D face reconstruction, preserving the patient‘s pathological features. 2. A face texture reconstruction method based on RGBD sequences is designed and implemented. This method first utilizes a frame selection strategy to avoid blurred facial images in the selected frames. It then applies delighting techniques to the selected frames to solve the problem of texture reconstruction under extreme lighting conditions. Finally, the method uses a Laplacian pyramid for texture fusion to achieve high-precision facial texture reconstruction. 3. A set of auxiliary diagnosis system integrating data acquisition, 3D face reconstruction and online visualization and evaluation is designed and implemented. This system supports a complete workflow for remote 3D facial diagnosis and evaluation, including voice and visual guidance for patient data collection, data upload, cloud-based computation, and web-based result presentation.