视觉感知在移动机器人的感知系统中扮演至关重要的角色，为了满足现今移动机器人应用于复杂环境实时性、泛化性、高精确性的需求，多传感器的联合应用识别技术是关键，现有视觉感知系统的主流传感器为RGB-D相机与激光雷达，因为具有能直接获取环境三维信息的能力而被广泛使用。因此本文意在有效地应用RGB-D相机与激光雷达感知系统，满足在复杂环境中所需的感知能力，并针对系统整合与目标识别算法进行深入研究;同时考虑识别过程中，由于目标物遮挡以及目标物过远等情况导致的低识别可信度的情况，本文拟通过主动视觉算法改变感知系统位姿用以补充目标物信息，最终与目标识别算法构成闭环的目标检测算法提升识别结果的可信度。首先，外参矩阵是应用RGB-D相机与激光雷达数据的基础，本文在充分调研现有标定算法的基础上，提出一种基于各向异性的外参矩阵标定算法。在实际使用传感器时，发现传感器测量的原理会导致数据误差呈现各向异性，由于现有算法皆为基于特征点迭代，若迭代点呈各向异性分布将导致标定精度下降。针对各向异性问题，本文通过分析RGB-D相机与激光雷达的测量原理并建构加权矩阵，降低各向异性对标定算法的影响，最终提高外参矩阵的标定精度。其次，面向高实时性、高泛化性的环境感知能力，本文研究了基于深度学习的目标识别算法。针对算法在识别过程存在的低实时性问题，通过优化算法架构提高算法实时性，并制定标记目标物规则，建立自定义数据集，通过在自定义数据集的实验，验证优化后的网络架构能有效降低目标识别算法的耗时，同时验证该算法具高泛化性。最后，为实现高精确性的目标检测能力，针对识别结果低可信度的特殊情况，首先对主动视觉算法进行研究，提出了结合深度信息与RGB信息的一种基于强化学习的主动视觉算法，相较于传统算法，本算法仅需通过当前状态即可预测最佳视野位姿，通过移动机器人位置，获取新的目标物信息，克服信息不足的问题。实验证明本文提出的强化学习主动视觉算法能有效地将感知系统移动至最佳视野位置，提升目标识别算法识别结果的可信度。

Vision plays a significant role in mobile robots’ perception system. Multi-sensor system is the key to modern mobile robots which have the requirements on real-time, robustness and high accuracy. Because RGB-D cameras and LiDARs can obtain three-dimensional data directly, they are widely used. This paper aims apply RGB-D camera and LiDAR system efficiently to meet the requirements of using robots in complex environment. This paper also delves into combining sensors and object detection. Meanwhile, for the low confidence case which are due to occlusion or distance during the detection process, we intend to change the position of perception system with the active vision algorithm to acquire new data of object. Finally, active vision and object detection form a closed-loop detection method to increase the confidence of recognition results.First, calibrating the extrinsic matrices between sensors is a significant pre-processing step of multi-sensor system. Consequently, after investigating existing calibration methods, this paper proposes a calibration method with anistropic weighting for LiDAR and stereo camera system. Because of the internal measurement properties of sensors, the error distribution of point coordinates is anisotropic. However, most of existing calibration methods use point-based rigid registration algorithm which leading to decrease calibration accuracy. To anistropic issue we construct a weighting matrix which is based on the measurement properties of sensors. Eventually, this method decreases the influence of anisotropic, and increases the accuracy.Secondly, for the perception ability with real-time and robustness, this paper studies object detection algorithm which is based on deep learning. To the low real-time issue of existing algorithms, this paper increases the effectiveness of algorithm by replacing backbone network. At the same time, we make the rules of labeling images to build the custom dataset. On the custom dataset, the experiments verify that the optimized algorithm can reduce the time consumption, and have high robustness.Finally, in order to increase the accuracy of object detection in low confidence cases, this paper researches into active vision algorithm, and proposes an algorithm with RGB-D data based on reinforcement learning. Comparing with traditional algorithm, the proposed method only needs current state to predict a new location to get new information of object, and overcome the difficulties of lack of data. The experiments verify that the proposed algorithm can move the perception system to the location with the best view, to increase the confidence of detection result.