互联网的快速发展对网络的运营和管理提出了更高的要求。流量工程（Traffic Engineering，TE）可以通过优化路由来提升网络性能和资源利用率，因而一直是学术界和工业界研究的热点。为应对日益复杂的网络环境，当前研究需要进一步加强流量工程方案的网络状态自适应能力，以高效地实现服务质量（Quality of Service，QoS）提升、负载均衡等流量工程目标。 本文对现有流量工程研究进行了充分调研，并从接入、域内和域间三个角度进行了网络状态自适应的流量工程研究。本文主要研究内容和贡献如下： 1. 提出了路径时延约束的车联网流量工程方案。设计了一个具有理论性能保证的强化学习算法，该算法根据数据速率和传输时延这些信息来优化车辆的下行数据传输路径，在提升整体传输速率的同时，满足峰值和平均路径时延约束。与传统的模型驱动的方案不同，所提方案借助数据驱动的方法对路由进行优化，能够更高效地适应网络状态的变化，满足应用复杂的QoS需求。仿真结果显示，所提方案将平均传输速率提升了至少40%，同时满足了峰值和平均时延约束。 2. 提出了流量特性驱动的域内网络流量工程方案。基于对真实数据的流量特性的分析和建模，从粗粒度路由和细粒度路由两个角度对网络流量感知和路由计算方式进行优化，以高效地应对网络流量的变化。首先提出了一个基于流量矩阵分类的粗粒度路由方案，包括设计聚类算法对历史流量矩阵聚类并计算相应的候选路由；构建机器学习分类器，它能够根据低开销的路由器端口统计数据在线地选择最佳候选路由。 之后进一步提出了一个基于大流调度的细粒度路由方案，包括对大流调度问题进行建模并证明该优化问题属于NP难；设计一个具有理论性能保证的随机算法为探知的大流分配转发路径。性能评价结果显示，上述方案可以高效地应对网络流量的变化、实现负载均衡且不会引入过多的开销。 3. 提出了链路状态感知的域间网络流量工程方案。设计了基于深度强化学习的分布式流量工程框架，各个区域的深度强化学习智能体可以在独立计算路由的情况下实现全局负载均衡的目标。本文为智能体设计了链路状态感知的输入、动作空间优化的输出以及优化目标相关的奖励函数，并采用增量式训练方法增强方案对网络流量和网络拓扑变化的适应能力。仿真结果显示，该方案可以在90%的测试中将全局的拥塞指标限制在 1.2 倍的最优拥塞指标下。
The rapid development of the Internet has put forward higher requirements for network operation and management. Traffic Engineering (TE) can improve network performance and resource utilization by optimizing routings, so it has been a hot research topic in academia and industry. In response to the increasingly complex network environment, the current research needs to further strengthen the TE schemes' ability of network state adaptiveness to effectively achieve the Quality of Service (QoS) improvement, load balancing, and other TE goals. This dissertation does a comprehensive survey on existing TE approaches. After that, we do research on the network state adaptive TE in access networks, intra-domain networks, and inter-domain networks. The main research contents and contributions are as follows: 1. We have proposed a path delay-constrained TE scheme for vehicular networks. A reinforcement learning (RL) algorithm with theoretical performance guarantees is designed. It optimizes the downlink data transmission paths of vehicles based on the information of data rate and transmission delay, which improves the overall transmission rate while satisfying peak and average path delay constraints. Different from the traditional model-driven schemes, the proposed scheme optimizes routings with the help of data-driven methods. This makes our scheme be able to adapt to the changes of network state more effectively and meet the complex QoS requirements of applications. Simulation results show that the proposed scheme improves the average transmission rate by at least 40% and meets both the peak and average delay constraints. 2. We have proposed a traffic characteristic-driven TE scheme for intra-domain networks. Based on the analysis and modeling of the traffic characteristics of real data, the network traffic perception and routing computation methods are optimized from the two aspects of coarse-grained routing and fine-grained routing to deal with network traffic changes efficiently. Firstly, a coarse-grained routing scheme based on traffic matrix classification is proposed. A clustering algorithm is designed to group historical traffic matrices and compute corresponding candidate routings for the obtained groups (or clusters). A machine learning classifier is constructed. It can select the best candidate routing online according to the low-cost statistics of router ports. Furthermore, a fine-grained routing scheme based on large flow scheduling is proposed. The large flow scheduling problem is modeled and proved to be NP-hard. A stochastic algorithm with theoretical performance guarantees is proposed to allocate forwarding paths for detected large flows. Evaluation results show that the above schemes can effectively cope with network traffic changes, achieve load balancing, and do not introduce too much overhead. 3. We have proposed a link state-aware TE scheme for inter-domain networks. A distributed TE framework based on deep reinforcement learning (DRL) is designed. The DRL agent in each region can achieve the global load balancing goal in the case of independent routing computation. In this paper, the link state-aware input, the action space-optimized output, and the optimization objective-related reward function are designed for the agents. The incremental training method is used to enhance the scheme's adaptability to network traffic changes and network topology changes. Simulation results show that the global congestion metric can be limited to 1.2 times the optimal congestion metric in 90% of tests.
