人脸识别由于其便捷性和准确性在交通、安防和金融支付等领域具有广泛的应用。但随着 3D 打印和仿生硅胶等先进材料技术的不断发展,高仿真人脸面具的出现让人脸识别系统面临着更加严峻的挑战。如何提高人脸鉴伪的准确性,是目前计算机视觉领域需要解决的一大核心问题。 相比于主流的基于 RGB 模态的人脸鉴伪方法,光谱成像技术能够引入额外的光谱信息,利用人脸皮肤独特的光谱特征取得更加准确的鉴伪结果。然而,目前已广泛应用的近红外传感器光谱特征有限;而传统的高光谱相机成本高昂、体积大且需扫描成像,难以集成到现有的人脸识别系统中。可实时获得压缩感知光谱图像的快照式光谱成像传感器使人脸鉴伪有了新的发展前景,但还面临着诸多挑战。一方面,这种新型传感器重建高光谱图像需要消耗大量的时间及计算资源,并且精度有限;另一方面,目前基于此技术的研究主要停留在光谱重建仿真层面,还没有实际应用于人脸鉴伪的方法。 本文设计并提出了基于快照式光谱成像技术的人脸鉴伪方法,建立了多材质面具鉴伪系统,实现了对各种材质高仿真面具的高效、准确鉴别。首先,本文借助快照式光谱相机建立了首个高光谱面具鉴伪数据集 HySpeMAS。然后初步提出了基于压缩感知光谱重建的人脸鉴伪方法,将快照式光谱成像技术首次实际部署至人脸鉴伪任务中。在此基础上,本文又进一步提出了基于神经网络的软硬件一体化鉴伪方法。软硬件一体化鉴伪方法直接将快照式光谱成像芯片作为光学编码器,并采用端到端的快照式光谱成像网络 SSINet 对光学编码结果进行解码得到鉴伪结果。软硬件一体化的鉴伪方法相比于压缩感知鉴伪方法,不再需要光谱重建,单张光谱图像的鉴伪时间从 5~10min 大幅缩短至 5~10ms,平均错误率 ACER 下降8.36%,并且相比于基于 RGB 图像最好的鉴伪方法 ACER 下降 8.43%。此外,针对特定的面部遮挡场景,本文提出了人脸不完整、特征缺失下的识别和鉴伪方案。 最后,本文基于快照式光谱成像的多材质面具鉴伪系统在 2023 年成都夏季世界大学生运动会期间进行了现场部署,在实地场景中完成了系统验证和应用示范,体现了所提出方法在高仿人脸鉴伪上的准确性、时效性和泛化性。
Facial recognition, due to its convenience and accuracy, has found widespread applications in transportation, security, and financial payment systems. However, with the continuous development of advanced material technologies such as 3D printing and biomimetic silicone, the emergence of highly realistic facial masks poses an even greater challenge to facial recognition systems. Improving the accuracy of facial anti-spoofing and applying it to daily facial recognition scenarios is a core issue that needs to be addressed in the field of computer vision. Compared to mainstream RGB-based facial anti-spoofing methods, spectral imaging technology introduces additional spectral information and achieves more accurate anti-spoofing results by utilizing the unique spectral characteristics of human facial skin. Nevertheless, the spectral features of the widely used near-infrared sensors are limited; traditional hyperspectral cameras are expensive, bulky, and require scanning to image, making them difficult to integrate into existing facial recognition systems. The snapshot spectral imaging sensor, which can obtain compressed-sensing spectral images in real-time, presents new opportunities for the development of facial anti-spoofing, yet it faces several challenges. On one hand, this novel sensor requires considerable time and computational resources to reconstruct hyperspectral images, and the accuracy is limited. On the other hand, current research based on this technology is primarily at the stage of spectral reconstruction simulation, with no practical applications in facial anti-spoofing methods as of yet. This paper designs and proposes a facial anti-spoofing method based on snapshot spectral imaging technology, establishing a multi-material mask anti-spoofing system that achieves efficient and accurate identification of various highly realistic masks. Firstly, this paper utilized a snapshot spectral camera to establish the first Hyperspectral Mask Anti-spoofing Dataset, HySpeMAS. Subsequently, it preliminarily proposed a facial anti-spoofing method based on compressed sensing spectral reconstruction, which marks the first practical application of snapshot spectral imaging technology to facial anti-spoofing tasks. On this basis, this paper proposes a hardware-software integrated anti-spoofing method based on neural networks. The hardware-software integrated method directly uses the snapshot spectral imaging chip as an optical encoder and employs an end-to-end Snapshot Spectral Imaging Network (SSINet) to decode the optical encoding results for anti-spoofing. Compared to the compressed sensing anti-spoofing method, the hardware-software integrated method no longer requires spectral reconstruction, reducing the anti-spoofing time for a single spectral image from 5-10 minutes to 5-10 milliseconds, with an average error rate (ACER) decrease of 8.36%, and a decrease of 8.43% compared to the best anti-spoofing method based on RGB images. In addition, for specific facial occlusion scenarios, this paper proposes recognition and anti-spoofing solutions for incomplete faces and missing features. Finally, the multi-material mask anti-spoofing system based on snapshot spectral imaging established in this paper was deployed on-site during the 2023 Chengdu Summer World University Games, completing system verification and application demonstration in real-world scenarios, demonstrating the accuracy, timeliness, and generalizability of the proposed method in high-fidelity facial anti-spoofing.