绕机检查是民航公司起飞前所做的关乎航空安全的重要准备工作之一，机组人员需要根据具体飞机型号的绕机检查说明手册，按照一定顺序和标准完成检查。目前业界大多依赖检查人员的个人专业素质及工作经验，检查质量容易受到个体因素、天气状况等其他外界因素干扰，无法对检查质量做有效保证。近年来增强现实及机器学习技术的发展，对这一场景的改进提供了新的可能性。将机器学习算法注入眼镜式AR穿戴设备，让工作人员在检查过程中及检查后得到检查质量的评分，及时识别因环境因素及人为因素造成的检查质量不佳，对于保障机组安全运行和民航事业发展都有着重要意义。结合研究资源与研究设计，本文通过眼动仪收集的某民航公司机组人员绕机检查的167段第一视角视频，检查过程中眼动、注视过程及瞳孔大小的时间序列特征，以及问卷收集的检查人员基本信息，采用视觉转换器（Vision Transformer）和长短期记忆（LSTM）算法提取序列特征，检查过程中每一时间帧对应的眼动、注视、凝视点、瞳孔等维度的结构化特征，借助专家打分作为标签，通过系统的特征工程，构建了基于多核支持向量机（Multi-Kernel Support Vector Machine）的检查质量识别（分类）模型。并使用适合本数据集的聚类算法对不同的检查模式进行了进一步的探索。本研究通过和逻辑回归、单一核函数的支持向量机等基准模型比较表明多核融合的支持向量机在召回率、F1得分以及AUC指标中均具有更出色的表现（召回率0.80, F1分数0.84, AUC结果0.86），且具有一定的稳健性。本研究希望据此改进AR眼镜的设计，为智能化判断绕机检查质量乃至标准化巡检场景提供可以实践落地的改进。本研究对于小样本非结构性数据的处理，以及多模态数据的特征融合与使用，做出了一定的创新性探索。研究结果扩展了增强现实技术的落地场景，提供了新的眼动特征与认知行为的实证证据，为绕机检查业界实践提供了一定的指导意义。
One of the vital steps of safe flight preparation is a “walk-around”, which is also called a pre-flight inspection conducted by the flight crew. The crew is responsible to follow the guide book of the specific flight model to conduct inspection with an exact order and standard.However, most of the civil aviation companies relies on the employees’ personal professional experience to conduct the inspection, thus the results are of high stake to be affected by individual factors, weather conditions and others, making it hard to guarantee the inspection quality. Equipping the inspection process with AR goggles is an emerging way to enhance the safety flight. Based on egocentric inspection videos and eye-tracking devices, a multi-modality model could be designed to help recognize unsatisfying inspections. This research will have profound significance for not only the safe operation of the flight crew, but for the long-term development of China civil aviation. Based on the methodology from systematic literature review, eye-tracking devices were used to collect 167 egocentric videos of pre-flight inspection. Besides, the researcher collected the eye movement, fixation, gaze point, pupil and other features corresponding to each frame in the inspection process using the eye-tracker. Vision transformer and LSTM algorithm were used to extract sequence features and structural features of the videos. With expert scoring as the label, an inspection quality recognition (classification) model based on multi-kernel support vector machine was designed. This study did the model evaluation with logistic regression, support vector machine (SVM) with single kernel function, support vector machine (SVM) with single modality data and single kernel function as base models. It turned out that the multi-kernel fusion support vector machine had better performance in recall rate (0.80), F1 score (0.84) and AUC metrics (0.86), and has nontrivial robustness. This study made some innovative explorations on the processing of small samples, unbalanced and unstructured data using feature fusion and multi-modality methods. The results of this study extended the practical usage of augmented reality technology, provided new empirical evidence of eye movement characteristics and cognitive behavior, and provided certain guiding significance for the industry practice of flight inspection.
