三维点云是目前主流的三维物体表示方式。近年来随着深度学习的发展,基于深度神经网络的点云分析方法已经在各种三维应用中获得巨大的成功。这些成功多来源于对神经网络结构的深入改进,而较少关注对输入点云的质量优化。例如基于三维点的网络直接使用从物理设备中扫描获得的点云作为输入,然而这些点云往往存在缺失和噪声等问题;基于点云图像的网络使用渲染的点云图像作为输入,然而这种点云图像也存在三维形状的缺失。这些问题导致网络输入的数据质量较低,难以反映出物体的真实形状,限制了网络的性能。本文聚焦于结合低质量点云的形状先验来研究高质量点云表示的生成,并将生成的高质量的表示应用于各种点云任务以提升性能。本文的主要研究内容和贡献包括:1) 对于真实的三维点云,针对其存在的缺失、稀疏和噪声等问题,分别提出了形状先验驱动的点云补全和任意倍率上采样网络框架。通过几何形状先验驱动的补全网络,将存在缺失的点云根据其局部几何形状先验补全为完整的点云。针对稀疏和不均匀的点云,通过形状先验驱动的任意倍率上采样网络将其转换为稠密且均匀的点云。最后将训练好的补全和上采样网络统一到一个高质量点云生成与应用框架中,该框架由生成模块和应用模块构成。在生成模块中依次集成训练好的补全和上采样网络生成高质量的三维点云。在应用模块中集成基于三维点的下游任务网络,通过端到端的训练,将生成的高质量点云应用到各种任务中提升性能。2) 对于点云的图像表示,针对现有渲染过程中由于维度降低导致的几何信息损失的问题,提出一种语义感知的可微的特征渲染方法。首先设计基于点的语义感知网络从三维点云中提取感知形状先验的点级特征,之后通过可微渲染模块将提取的具有更多点云几何形状信息的特征渲染到点云图像中来减少点云图像的信息损失。通过将该方法集成到现有的基于点云渲染图像的各种下游任务网络中并执行端到端的训练,使得梯度从图像域传递到基于点的语义感知网络中,生成含有丰富形状先验且面向各种任务的点云图像以提升任务的性能。
Three-dimensional point clouds are currently the mainstream representation for three-dimensional objects. In recent years, with the development of deep learning, point cloud analysis methods based on deep neural networks have achieved significant success in various three-dimensional applications. Much of this success stems from profound improvements in neural network architectures, with less attention paid to optimizing the quality of input point clouds. For example, point-level networks directly utilize point clouds obtained from physical devices as input; however, these point clouds often suffer from issues such as missing data and noise. Point cloud image networks utilize rendered point cloud images as input; however, such point cloud images also exhibit deficiencies in representing three-dimensional shapes. These issues result in lower data quality input to networks, making it difficult to accurately reflect the true shape of objects and thus limiting network performance. This thesis focuses on integrating shape priors of low-quality point clouds to investigate the generation of high-quality point cloud representations and apply these generated high-quality representations to various point cloud tasks to enhance performance. The main research content and contributions of this paper include:1) For real-world three-dimensional point clouds, addressing issues such as missing data, sparsity, and noise, we propose shape prior-driven point cloud completion and arbitrary-rate upsampling network frameworks. With the geometric shape prior-driven completion network, missing points in the point cloud are completed based on their local geometric shape priors. For sparse and unevenly distributed point clouds, the shape prior-driven arbitrary-rate upsampling network transforms them into dense and uniformly distributed point clouds. Finally, the trained completion and upsampling networks are integrated into a high-quality point cloud generation and application framework, consisting of generation and application modules. In the generation module, the trained completion and upsampling networks are sequentially integrated to generate high-quality three-dimensional point clouds. In the application module, downstream task networks based on three-dimensional points are integrated, and through end-to-end training, the generated high-quality point clouds are applied to various tasks to enhance performance.2) For point cloud image representation, we propose a semantic-aware differentiable feature rendering method to address the issue of geometric information loss caused by dimensionality reduction in existing rendering processes. Firstly, a point-based semantic-aware network is designed to extract point-level features capturing the perceived shape priors from three-dimensional point clouds. Subsequently, these extracted features, enriched with more geometric information of the point cloud, are rendered into point cloud images through a differentiable rendering module to mitigate information loss. By integrating this method into various downstream task networks based on point cloud image rendering and conducting end-to-end training, gradients are propagated from the image domain to the point-based semantic-aware network. This process generates point cloud images with rich shape priors tailored to various tasks, thereby enhancing task performance.