强化学习作为一类数据驱动的控制方法，在工业控制领域潜力巨大。然而工业控制问题的强非线性与高复杂性给强化学习应用带来挑战：其值函数评估准确度低、稳定性差，且策略输出不平滑，严重影响控制性能。针对上述难题，本文提出了具有过估计抑制能力的低方差值分布函数更新机制，设计了具有低通滤波特性的神经网络策略结构，并开发了面向工业控制的通用强化学习求解软件。通过自动驾驶仿真与实车测试系统验证了所提方法的有效性，为强化学习在工业控制问题中的应用提供理论与实践支撑。 首先，针对强化学习值函数评估准确性低、收敛不稳的问题，提出了一种具有过估计抑制能力的低方差值分布函数更新机制。通过推导不依赖目标值采样的均值梯度更新公式构建低方差优化目标，提升值分布估计精度。设计基于值分布标准差的梯度调整机制改善值分布估计稳定性，并引入对称值分布更新设计进一步抑制过估计。结合最大熵原理提出高稳定特性的值分布强化学习算法。仿真实验表明其最大性能提升16.0%，值函数估计准确度在所有任务中处于领先水平。 其次，针对神经网络策略易受噪声干扰、输出不平滑的问题，构建了具有低通滤波特性的神经网络结构。通过设计具有一阶惯性滤波特性的惯性神经元与可动态调整惯性神经元平滑强度的自适应神经元，实现策略平滑性与控制性能的有效平衡。仿真实验表明：网络结构在噪声环境下平均累积回报最大提升89.1%，表征平滑性能的动作震荡率与平均加权频率指标最大降低78.7%与34.4%。 然后，为满足强化学习多样化的工业应用需求，开发了通用强化学习求解软件GOPS。软件采用模块化分层解耦架构，通过统一抽象基类与接口设计，实现对13种异构主流强化学习算法的无缝兼容。设计多进程并行采样与分布式梯度计算机制，构建高效并行训练框架。通过在机器人控制等4种典型工业控制问题与8种代表性算法的综合功能测试，验证了GOPS在求解精度与训练效率方面的优越性能。测试结果显示策略动作误差低于1.7%，并行训练实现近线性加速效果。 最后，开发了城市混合交通流仿真环境，设计互注意力机制状态编码网络进行周车信息表征，建立自动驾驶决控的平滑型强化学习训练方案。仿真显示决控策略任务成功率最大提升32.9%，动作震荡率降低50.0%。将策略部署于实车平台，在包含7个信控路口的开放城市道路进行测试，策略在高复杂度真实环境下实现了平稳安全的决策控制，为强化学习在复杂工业控制领域的应用推广奠定基础。

Reinforcement learning, as a data-driven control method, holds tremendous potential in the field of industrial control. However, the strong nonlinearity and high complexity of industrial control problems pose significant challenges to reinforcement learning applications: low accuracy and poor stability in value function estimation, along with non-smooth policy outputs, severely affecting its control performance. To address these challenges, this dissertation proposes a low-variance value distribution function update mechanism with overestimation suppression capability, designs a neural network policy structure with low-pass filtering characteristics, and develops a general-purpose reinforcement learning solver software for industrial control applications. The effectiveness of the proposed methods is validated through autonomous driving simulation and real vehicle testing, providing theoretical and practical support for the application of reinforcement learning in industrial control problems. First, to address the issues of low accuracy and unstable convergence in reinforcement learning value function estimation, a low-variance value distribution function update mechanism with overestimation suppression capability is proposed. A low-variance optimization objective is constructed by deriving a mean value gradient update formula that does not rely on target value sampling, thereby improving the accuracy of value distribution estimation. A gradient adjustment mechanism based on value distribution standard deviation is designed to enhance the stability of value distribution estimation, and a symmetric value distribution update design is introduced to further suppress overestimation. Combined with the maximum entropy principle, a value distribution reinforcement learning algorithm with high stability characteristics is proposed. Simulation experiments demonstrate a maximum performance improvement of 16.0\%, with value function estimation accuracy leading across all tasks. Second, to address the issues of non-smooth neural network policy outputs and susceptibility to noise interference, a neural network structure with low-pass filtering characteristics is constructed. By designing inertial cell with first-order inertial filtering properties and adaptive cell that can dynamically adjust the smoothing strength of inertial cells, an effective balance between policy smoothness and control performance is achieved. Simulation experiments show that the network structure achieves a maximum improvement of 89.1\% in total average return under noisy environments, with action fluctuation rate and mean weighted frequency indicators representing smoothness performance reduced by a maximum of 78.7\% and 34.4\%, respectively. Then, to meet the diverse industrial application requirements of reinforcement learning, a general-purpose reinforcement learning solver software is developed. The software adopts a modular hierarchical decoupling architecture and achieves seamless compatibility with 13 heterogeneous mainstream reinforcement learning algorithms through unified abstract base classes and interface design. Multiprocess parallel sampling and distributed gradient computation mechanisms are designed to construct an efficient parallel training framework. Through comprehensive functional testing on four typical industrial control problems including robot control and eight representative algorithms, the superior performance of the software in solution accuracy and training efficiency is validated. Test results show policy action errors below 1.7\% and near-linear acceleration effects achieved through parallel training. Finally, an urban mixed traffic flow simulation environment is developed, a cross attention encoding network is designed for surrounding vehicle information representation, and a smooth reinforcement learning training scheme for autonomous driving decision and control is established. Simulations show that the decision and control policy achieves a maximum improvement of 32.9\% in task success rate and a 50.0\% reduction in action fluctuation rate. The policy is deployed on a real vehicle platform and tested on open urban roads containing 7 signalized intersections, where the policy achieves stable and safe decision-making control in highly complex real environments, laying the foundation for the application and promotion of reinforcement learning in complex industrial control fields.