登录 EN

添加临时用户

面向窗口六自由度的全景光场拼接算法研究

Research on Windowed-6DoF Panoramic Light Field Stitching

作者:周思瑶
  • 学号
    2019******
  • 学位
    硕士
  • 电子邮箱
    zho******.cn
  • 答辩日期
    2022.05.13
  • 导师
    金欣
  • 学科名
    控制工程
  • 页码
    95
  • 保密级别
    公开
  • 培养单位
    025 自动化系
  • 中文关键词
    光场拼接,窗口六自由度,多同心球面模型
  • 英文关键词
    Light field stitching, Windowed-6DoF, Multi concentric spherical modeling

摘要

窗口六自由度全景光场包含尺度受限的三维旋转与三维平移,能够将大视场空间光线的强度和方向信息重现在用户面前,带来身临其境的观看体验,在虚拟现实领域受到广泛关注。手持式光场相机因具备同时采集光线的强度和方向信息的能力而成为采集窗口六自由度光场的理想设备,但是由于其有限的视场角以及窗口六自由度的平移和旋转动作需求,使得研究窗口六自由度全景光场拼接,解决平移与旋转动作参数化、减小多光场拼接累积误差的问题成为必须。因此本文围绕以上问题展开研究,提出相应的解决方案,获取高质量的窗口六自由度全景光场拼接结果。 为了减小三维旋转带来的投影畸变,本文首先提出面向三自由度旋转光场的参数化模型与拼接方法。基于三自由度旋转动作分析,结合球面坐标系与四维光场的数据特点,提出同心球面模型实现角度域一致的旋转光场球面投影,然后基于贪心算法估计多光场拼接顺序,进而按顺序配准光场,显著降低了由于光场之间的旋转动作导致的投影误差,达到高精度拼接三维旋转光场的效果。 为了提高光场拼接对三维平移视差的鲁棒性,本文提出面向三自由度旋转与平移的光场参数化模型与拼接方法。针对平移,进一步拓展同心球面为多同心球面模型,单个同心球面描述三自由度旋转,同心球面之间的视差描述三自由度平移。然后从稀疏的对角线子孔径提取四维光场特征,并分深度地估计全局-局部单应性矩阵配准多光场,显著提升了光场配准精度和对旋转及平移视差的鲁棒性。 进一步,为了降低窗口六自由度多光场拼接的累积误差,本文提出基于光场特征簇聚类的窗口六自由度光场拼接。基于累积误差来源分析,提出树结构拼接顺序,将每个光场视为叶子节点,根据特征点对数量聚类光场簇,簇内光场的拼接结果为父节点,逐层向上拼接得到窗口六自由度光场拼接结果,显著减小了由累积误差导致的畸变错位。最后,本文设计了窗口六自由度全景光场演示系统,在头戴式显示器中沉浸式展示窗口六自由度全景光场。 本研究利用窗口六自由度动作特点和光场结构特性,实现面向窗口六自由度的全景光场拼接,获得超大视场光场并应用于虚拟现实展示,为光场在计算机视觉领域的应用做出贡献。

Windowed-6 degrees of freedom (Windowed-6DoF) light field (LF) supports restricted 3DoF rotation and 3DoF translation, and can reproduce the intensities and directions of the light rays in a wide space for viewers, to provide immersive viewing experiences, so it has attracted the attention of virtual reality (VR) field. Hand-held plenoptic camera can simultaneously record the intensity and direction information of light, so that has become an ideal device to capture 6DoF LFs. However, due to the limited field of view (FOV) of the hand-held plenoptic camera and the 6DoF translation and rotation requirements of windowed-6DoF, the research on windowed-6DoF LFs stitching is necessary, which would solve the parameterization of rotational and translational motion and reduce the cumulative error of multiple LFs stitching. Therefore, this paper focuses on the above problems and puts forward corresponding solutions to obtain high-quality windowed-6DoF panoramic stitching results. In order to reduce the projection distortion caused by 3DoF rotation, a 3DoF rotational LF stitching algorithm is proposed. Firstly, based on the analysis of 3DoF rotational motion, and combined with characteristics of the spherical coordinate system and 4D LF, a concentric spherical model is established to realize the spherical projection of the 3DoF rotating LFs in a consistent angle domain. Then, the stitching order of multiple LFs is estimated based on a greedy algorithm, and then the LFs are registered in order. The proposed algorithm significantly reduces the projection error caused by the rotation motion between adjacent LFs, stitching 3DoF LFs with high precision. In order to improve the robustness of LF stitching to the parallax caused by 3DoF translation, a 3DoF rotational and translational LF stitching algorithm is proposed. For translational motion, the concentric spherical model is further extended to be a multi-concentric spherical model, in which a single concentric sphere describes the 3DoF rotation, and the parallax between concentric spheres describes the 3DoF translation. Then, given the 4D light field feature (LiFF) on diagonal sub-aperture images, chained global-local adaptive homographies based on depth layer maps (DLMs) are constructed to register multiple LFs. The proposed algorithm significantly improves the light field registration accuracy and robustness to rotation and translation parallax. Furthermore, in order to reduce the cumulative error of windowed-6DoF multiple LFs stitching, this paper proposes a windowed-6DoF multiple LFs stitching algorithm based on 4D LiFF clustering. Based on the analysis of cumulative error sources, the tree structure stitching order is proposed, in which each input LF is regarded as a leaf node, and LFs are clustered according to the number of feature point pairs. The LFs in each cluster are stitched in order to obtain their parent node. Then, the panoramic LF stitching result in the root node is obtained by stitching layer by layer. The proposed algorithm significantly reduces the distortion and dislocation caused by cumulative error. Finally, this paper designs a windowed-6DoF panoramic LF demonstration system and immerses the windowed-6DoF panoramic LF in the head-mounted display. The proposed LF stitching algorithm implements robust multiple LF stitching in windowed-6DoF space, which obtains large FoV or even panoramic LF and applies it to VR display, which makes a contribution to the application of LF in computer vision.