随着近年来光电成像系统的小型化发展与大范围使用，许多传统光学现象的研究和应用都面临着全新的机遇与挑战。猫眼效应，作为成像设备所特有的逆向反射性质，已经被应用于光电探测、空间光通信等诸多领域，但同时亦有很多新的问题亟待解决。本论文围绕着猫眼效应这一光学现象，从光电成像系统的光束传播分析出发，发现了具有实际应用的规律现象，解决了之前存在的难点问题，并通过实验和设计实例展开了深入的研究。主要的工作和创新点包括以下方面： 1.发现并验证了以CCD相机为代表的阵列成像系统中光敏面像素的周期性排布对于猫眼效应反射光束的影响规律。首先分析了猫眼效应的光束反射传播过程，其中充分考虑到了光敏面的反射率分布及衍射情况。由此，针对阵列成像系统的猫眼效应进行了数值仿真和实验验证，发现了其猫眼效应反射光束截面分布的周期性特点。并且进一步推导、仿真和实验得到了反射光束截面在会聚点的空间频率与猫眼目标自身参数之间的关系表达式。由此即可从猫眼效应的反射光束中提取出光电成像系统的内部信息; 2.搭建了基于猫眼效应的主动激光探测系统，实现了对运动光电成像设备的实时探测识别。首先计算评估了猫眼效应的探测性能，接着归纳提出了猫眼目标反射光斑的图像特征。基于以上，设计搭建了硬件样机平台，并针对不同类型猫眼目标提出了对应的识别算法，实现了运动猫眼目标的实时确认和追踪。经过实验验证，该系统可适用于识别和对抗监视无人机这类动态的成像设备，也对相机和枪瞄镜等普通静止监视目标有着准确的探测识别效果； 3.解决了抑制猫眼效应的同时保持光电系统成像质量这一难题，提出了三种解决方法且均完成了实际光学设计。这三种方法都在原有光学系统内加入了相位元件。其中相位板优化设计法对猫眼效应和成像信噪比同时进行约束，利用点扩散函数解得相位板面型，得到了较好成像质量且将猫眼效应降低两个数量级。自由曲面镜设计法通过横向像移的要求将反射光束阻隔在光阑之外，并由此要求设计优化得到自由曲面镜，实现了接近衍射极限的极好成像质量下消除猫眼效应。计算成像法通过波前编码将猫眼效应削弱三个数量级以上，再利用轻量化机器学习算法将图像复原得到好的成像质量。这三种技术方案均能有效解除猫眼效应带给成像系统易暴露的隐患，实现光电隐身。

With the miniaturization and wide application of photoelectric imaging systems in recent years, many researches and applications on traditional optical phenomena are facing new opportunities and challenges. Cat-eye effect, as a unique retroreflective characteristic of imaging devices, has been applied to many aspects such as photoelectric detection and free space optical communication, but there are also many new problems to be solved. Focusing on cat-eye effect, this dissertation starts from the beam propagation analysis of photoelectric imaging system. We find some optical phenomena with practical application, solve the previous tricky problems, and demonstrate new approaches with experiments and design examples. The main contributions include the following: \begin{itemize} 1. We find and verify the great influence of the reflectivity distribution at the image plane of the photoelectric imaging systems (e.g., CCD camera) on the cat-eye effect reflected beam profiles. Firstly, an applicable propagation model of cat-eye effect is proposed, while both the reflectivity distribution of the photosensitive surface and the internal diffraction are fully considered. Based on this model, the cat-eye effect of the optical system with sensor array is numerically simulated and experimentally verified. The periodical characteristics of the cat-eye effect reflected beam profiles are found. Furthermore, the relationship between the spatial frequency of the reflected beam profiles at the convergent point and the intrinsic parameters of cat-eye target is derived, simulated and experimentally derived. Thereby, the internal information of the photoelectric imaging system can be extracted from the cat-eye reflected beam profiles; 2. We establish an active laser detection system based on cat-eye effect, and the real-time detection and recognition of the moving photoelectric imaging devices are realized. Firstly, the detection ability of cat-eye effect is calculated and evaluated, and then properties of reflected spots generated by cat-eye targets are investigated. Therefore, the hardware prototype platform is designed and built, and corresponding recognition algorithms are proposed for different types of cat-eye targets. The approach realizes the real-time confirmation and tracking of the moving cat-eye targets. Experiments verify that the system can be applied to tracking and confronting moving surveillance devices such as UAVs, and also has an accurate detection and recognition performance on common static monitoring targets including cameras and optical sights; 3. We solve the tricky problem of reducing cat-eye effect while maintaining the imaging quality of the photoelectric system, and three different approaches are proposed and the optical lens design is completed. All three methods incorporate optical phase elements into the original optical system. Among them, the optimized phase mask design method constrains the cat-eye effect and the imaging signal-to-noise ratio at the same time, and solves the surface function of the phase mask by point spread function. This approach obtains preferable image quality and reduces cat-eye effect by two orders of magnitude. The freeform lens design method blocks the reflected beam outside the optical aperture by the requirement of transverse image translation. According to the translation requirement, a freeform single lens comprised of two different freeform surfaces is designed, which can achieve excellent imaging quality close to the diffraction limit while eliminating the cat-eye effect. The computational imaging method weakens the cat-eye effect by more than three orders of magnitude by wavefront coding, and then uses a light-weight machine learning algorithm to obtain good image quality. All three technical solutions can effectively eliminate threats to photoelectric imaging devices from the active laser detection systems due to the cat-eye effect, and realize photoelectric stealth.