增强现实辅助装配技术(简称增强装配或AR辅助装配)是一种将虚拟的装配信息与真实装配环境相结合的技术。这种技术可以提高装配信息传达的准确性和效率,进而提升操作人员的工作效率,并降低装配错误率和学习难度。但目前该领域在装配仿真虚拟场景生成一致性、装配引导指令设计标准化和生成自动化、装配检测及反馈能力等方面仍存在很多不足,无法满足航天航空等领域的大型、复杂装配体的应用场景。本文针对增强现实辅助装配中仿真、引导和检测三个阶段智能化的关键技术进行研究,聚焦增强装配场景注册及生成、AR远程协同装配训练、增强装配引导指令自动生成、装配要素位置动态检测和装配质量一致性检测5项主要技术,开发覆盖装配全阶段且智能化程度高的增强现实辅助装配系统。首先,针对装配环境的特点提出了基于直接法和复合ICP及PnP的三维注册算法以及AR多空间协同及匹配方法和虚实场景一致性融合方法,保证了在AR多空间装配环境下虚拟场景及物体的准确、一致和快速的生成。进而提出了一种基于共享手势的远程协同装配训练方法,实现了在3D-AR环境下对远程专家手势高精度的捕捉和准确的复现。其次,在装配引导方面提出了一种基于最小化用户认知负荷的增强装配引导指令设计方法,将装配工艺模型中的各类结构化信息与唯一形式的标准可视化增强装配引导指令进行匹配,实现标准化的引导指令体系设计,进一步地基于XML和Unity3D实现增强装配指令的标准化、自动化生成。在装配检测方面,提出了一种在离线AR环境下基于深度学习的装配要素检测及空间三维标注方法,在操作者视野中显示装配要素的确切空间位置并进行标记,达到低时延、高精度的动态空间物体检测效果。同时,提出了一种基于AR虚实图像注意力机制的装配结果一致性检测方法,在AR空间中对装配仪器的装配位置和线缆的关键节点位置、敷设路径、弯曲半径等信息进行计算,实现对装配结果的智能化检测。最后,本文建立了ARATGDS(Augmented reality assembly training, guidance and detection system,增强现实辅助装配训练、引导及检测系统),针对火箭舱段装配过程进行了上述功能的系统应用验证,证明了各个模块提出方法和建立系统的有效性。
Augmented reality-assisted assembly (abbreviated as augmented assembly or AR-assisted assembly) is a technology that combines virtual assembly information with the real assembly environment. This technology can improve the accuracy and efficiency of communicating assembly information, which in turn improves operator efficiency and reduces assembly error rates and learning difficulties. However, there are still many shortcomings in this field, such as the consistency of assembly simulation virtual scene generation, the standardization and automation of assembly guidance instruction design, and assembly detection and feedback capabilities, which cannot meet the application scenarios of large and complex assemblies in the aerospace industry. This paper focuses on the key technologies of intelligent simulation, guidance, and detection in the three stages of Augmented Reality Assisted Assembly, including scene registration and generation, AR remote collaborative assembly training, automatic generation of assembly guidance instructions, dynamic assembly element position detection, and assembly quality consistency detection. A high-level intelligent augmented reality-assisted assembly system covering the entire assembly process is developed.Firstly, a 3D registration algorithm based on the direct method and composite ICP and PnP, AR multi-space collaboration and matching methods, and virtual-real scene consistency fusion methods are proposed for assembly environment characteristics, ensuring the accurate, consistent, and rapid generation of virtual scenes and objects in AR multi-space assembly environments. A shared gesture-based remote collaborative assembly training method is proposed, achieving high-precision capture and accurate reproduction of remote expert gestures in the 3D-AR environment.Secondly, a method for designing augmented assembly guidance instructions based on minimizing user cognitive load is proposed for assembly guidance. This method matches various structured information in assembly process models with a unique form of standardized visual augmented assembly guidance instructions, realizing the design of a standardized guidance instruction system. Furthermore, based on XML and Unity3D, the standardized and automated generation of augmented assembly instructions is achieved.In the assembly detection aspect, an offline AR environment-based deep learning assembly element detection and spatial 3D annotation method is proposed. It displays the exact spatial position of assembly elements in the operator‘s field of view and marks them, achieving low-latency and high-precision dynamic spatial object detection effects. Simultaneously, an AR virtual-real image attention mechanism-based assembly result consistency detection method is proposed, calculating the pose of assembly instruments, key node positions, laying paths, and bending radius of cables in AR space, realizing intelligent detection of assembly results.Finally, the ARATGDS (Augmented reality assembly training, guidance and detection system) is established, and the system application verification of the above functions is conducted for the rocket cabin assembly process, demonstrating the effectiveness of the proposed methods and established system for each module.