登录 EN

添加临时用户

基于LSTM和粒子群的公有云容器资源调度研究

Research on Public Cloud Container Resource Scheduling Based on LSTM and Particle Swarm Optimization

作者:闫硕
  • 学号
    2020******
  • 学位
    硕士
  • 电子邮箱
    m13******com
  • 答辩日期
    2024.05.23
  • 导师
    杨帆
  • 学科名
    工程管理
  • 页码
    79
  • 保密级别
    公开
  • 培养单位
    025 自动化系
  • 中文关键词
    容器调度;长短期记忆网络;粒子群算法;云资源管理
  • 英文关键词
    Container scheduling; LSTM; Particle Swarm Optimization; Cloud resource management;

摘要

随着云计算的发展,越来越多的企业将业务系统部署到公有云上运行和使用,而容器技术凭借灵活弹性、敏捷迭代的特点,更成为了业务系统上云后的首选基础设施。但业内对容器资源的管理尚缺乏普适性的管理方法和管理手段,关于如何通过有效的调度手段结合容器灵活的部署方式,实现容器资源的最大化利用与最优配置,实现资源成本的节约,业内仍缺乏成熟的方法论和标准的指导。本文以此为出发点,研究基于容器资源预测和调度的关键技术,并结合企业的实际云资源管理的运作流程进行应用,主要工作如下:(1)提出一种基于融合调度系统的公有云容器资源管理方法,采用技术和管理相结合的手段解决容器资源的管理和最优配置问题。在技术方面,通过实时监控和预测两种手段为容器调度提供预先决策,并且建立面向场景的调度功能。在管理方面,将融合调度系统与企业云管理的流程相结合,面向云资源测算、新应用部署、存量资源配置优化、存量资源回收四类场景进行了流程设计。(2)针对容器资源预测问题,采用LSTM神经网络进行资源利用率的预测,对CPU和内存利用率进行特征值的挖掘,通过实验验证最优的LSTM神经网络配置,并且相比于RNN和BP神经网络能够实现更好的预测效果。(3)针对容器资源调度问题,采用粒子群算法提出了一种容器调度模型,将所有节点、所有类型资源分配量与平均分配量的差值之和作为粒子群算法中适应度函数,通过实验验证融合调度系统相比于Kubernetes默认的调度框架,能够实现更优的资源配置和利用率的均衡。(4)采用融合调度模型对Z公司云资源管理流程进行优化和重构,通过实际的应用和实验,实现了面对不同场景的容器资源配置优化,实现了资源的节约。

With the development of cloud computing, more and more enterprises are deploying their business systems to run and use public clouds. Container technology, with its flexible and iterative characteristics, has become the preferred infrastructure for cloud based business systems. However, the management of Container resources in the industry often adopts traditional physical server management methods. There is still a lack of mature methodology and standard guidance on how to achieve maximum utilization and optimal configuration of container resources through effective scheduling methods combined with flexible container deployment methods, and save resource costs. This article takes this as a starting point to study the key technologies based on container resource prediction and scheduling, and applies them in conjunction with the actual operational process of cloud resource management in enterprises. The main work of this thesis is as follows:(1)Propose a public cloud container resource management method based on a fusion scheduling system, which combines technology and management to solve the management and optimal configuration problems of container resources. In terms of technology, real-time monitoring and prediction are used to provide pre decision-making for container scheduling, and a scenario oriented scheduling function is established. In terms of management, the integrated scheduling system was combined with the process of enterprise cloud management, and process design was carried out for four scenarios: cloud resource calculation, new application deployment, optimization of stock resource allocation, and stock resource recovery.(2)For the problem of Container resource prediction, LSTM neural network is used to predict resource utilization, and feature values are mined for CPU and memory utilization. The optimal LSTM neural network configuration is verified through experiments, and better prediction results can be achieved compared to RNN and BP neural networks(3) A container scheduling model is proposed using particle swarm optimization algorithm to address the problem of container resource scheduling. The sum of the differences between the allocation of all nodes and types of resources and the average allocation is used as the fitness function in the particle swarm algorithm. Through experiments, it is verified that the fusion scheduling system can achieve a better balance between resource allocation and utilization compared to the default scheduling framework of Kubernetes.(4) We optimized and restructured the cloud resource management process of Company Z using a fusion scheduling model. Through practical applications and experiments, we achieved optimization of container resource configuration for different scenarios, achieving resource conservation.