近年来,基于深度学习的人脸复原方法在复原效果和计算效率方面已经取得了巨大的优势。然而,在现实世界中,低质量的人脸图像往往经历了复杂且多样的退化,现有的针对单一退化去除的图像处理方法很难泛化到真实场景中的人脸复原任务中。此外,许多针对人脸复原的方法,由于缺少有效的先验信息及合理的网络结构设计,较难复原出逼真的面部细节并达到较高的面部保真度。因此,本文从先验引入和模型结构两个方面对基于深度学习的人脸复原方法进行了改进,以生成更真实的面部细节并保持人脸的身份信息。本文的主要工作如下: 本文提出了基于域对齐生成式人脸先验的盲脸恢复算法。该方法采用了复杂的退化模型来合成训练数据集,模拟现实世界中真实的退化场景。通过引入嵌入在预训练人脸生成网络中的生成式人脸先验来辅助人脸复原,设计域对齐的GAN反转分支来提高生成式人脸先验的质量,并利用特征融合机制从生成式人脸先验特征和低质量输入中提取多分辨率卷积特征,以保存全局面部细节和图像背景,重建更真实且细节忠实度更高的人脸图像。 为了产生更加忠实于输入的细粒度面部细节,本文进一步提出了基于矢量量化字典先验的人脸恢复算法。该方法通过从大量高清的人脸图像中提取包含高清面部细节的矢量量化(Vector Quantization, VQ)字典来帮助网络在复原过程中恢复出逼真的面部细节。本文通过设计合理的VQ码本大小来帮助消除退化和生成真实细节,并设计通道分离的自适应归一化机制(CS-SAN)来抑制量化操作导致的重复伪影,同时还保留了从VQ字典中生成的真实细节,进一步提高人脸保真度和忠实细节恢复的平衡。 最后,本文在低质量人脸识别和老电影人脸修复两个实际应用场景下,进一步验证了本文所提出的人脸复原模型的有效性和普适性,并设计了一个简单易用的人脸复原UI交互应用小程序使得研究成果有一定的应用价值。 本文基于目前深度学习的人脸复原算法存在的问题,从先验引入和模型结构两个方面进行了改进,大量的对比实验证明了所提方法的有效性,并在相应的应用场景下进一步验证了所提人脸复原算法在实际应用中的可行性,为人脸复原算法的进一步研究和应用提供有力的支持和指导。
Face restoration methods based on deep learning have achieved significant advantages in restoration effect and computational efficiency in recent years. However, due to the complex and diverse degradation that low-quality face images often undergo in the real world, existing image processing methods that target a single degradation removal are difficult to generalize to face restoration tasks in real scenes. Many methods for face restoration lack effective prior information and reasonable network structure design, making it difficult to restore realistic facial details and achieve high fidelity. In this paper, we explore prior information that can effectively help face restoration, and design a reasonable network structure to generate more realistic facial details and maintain the identity information of the face during the restoration process. The main contributions of this paper are as follows: Firstly, a blind face restoration network based on domain-aligned generative face prior is proposed, which uses a complex degradation model to synthesize training dataset to simulate real degradation scenarios in the real world. By introducing a generative face prior embedded in a pre-trained face generation network to assist face restoration, we design a domain-aligned GAN (Generative Adversarial Network) inversion branch to improve the quality of the generative face prior. We use the designed feature fusion module to progressively fuse extracted multi-resolution convolution features from the degraded inputs and the generative face prior features to restore the face image, enabling the network to preserve global facial details and image background to reconstruct more realistic and faithful face images. To generate finer facial details that are more faithful to the input during the restoration process, we further explore high-quality facial priors that can be effectively utilized and propose a face restoration model based on vector quantization dictionary prior. This method extracts a VQ (Vector Quantization) dictionary containing high-quality (HQ) facial details from a large number of HQ face images through face reconstruction, and uses the dictionary to help the network restore realistic facial details during the restoration process. By designing a reasonable VQ codebook size to help eliminate degradation and generate real details, and designing a Channel-Split Adaptive Normalization (CS-SAN) to suppress the artifacts caused by quantization operation while preserving the real details generated from the VQ dictionary, we further improve the balance between face fidelity and faithful detail restoration. Finally, we further verify the generalization ability and applicability of the face restoration model proposed in this paper in real scenarios such as low-quality face recognition and face restoration in old movies. We also design a simple and easy-to-use face restoration UI interactive application program to make the research results have certain application value. In this thesis, we have made improvements in the introduction of prior information and model structure regarding the existing problems of deep learning-based face restoration methods. A large number of comparative experiments have demonstrated the effectiveness of the proposed models, and their feasibility in corresponding application scenarios has been further verified, providing strong support and guidance for further research and application of face restoration methods.