登录 EN

添加临时用户

贝叶斯网络方法研究

Research on Bayesian Network Approach

作者:田凤占
  • 学号
    9951******
  • 学位
    博士
  • 电子邮箱
    tfz******net
  • 答辩日期
    2002.05.22
  • 导师
    石纯一
  • 学科名
    计算机应用技术
  • 页码
    121
  • 保密级别
    公开
  • 馆藏号
    D02024-20
  • 培养单位
    024 计算机系
  • 中文关键词
    贝叶斯网络;动态贝叶斯网络;Noisy-Or节点;Noisy-And节点;隐藏变量;EM 算法;进化算法;增量学习;理论求精
  • 英文关键词
    bayesian networks;dynamic bayesian networks;noisy-or nodes;noisy-and nodes;hidden variables;EM algotithm;evolutionary algorithms;incremental learning;thoery refinement

摘要

贝叶斯网络是一种建立在概率和统计理论基础上的数据分析和辅助决策工具,以其坚实的理论基础、自然的表示方式、灵活的推理能力和方便的决策机制受到越来越多研究学者的重视。目前,贝叶斯网络已经广泛应用于数据挖掘、医疗诊断、工业控制、投资风险分析、预测、语音识别等领域,具有非常广阔的研究前景。贝叶斯网络的一个研究热点是从数据中学习贝叶斯网络的结构。目前,从完整数据中学习贝叶斯网络的结构已经有比较成熟和完善的方法,而从不完整数据集中学习贝叶斯网络的结构仍然是一个比较困难的问题。尤其是当网络结构中存在隐藏变量时,这个问题变得更加突出。有关贝叶斯网络的另一个研究热点是面向应用的特殊贝叶斯网络的研究。这些特殊贝叶斯网络包括:面向对象的贝叶斯网络、树状贝叶斯网络、多模块贝叶斯网络、Noisy-Or和Noisy-And节点组成的贝叶斯网络、动态贝叶斯网络等等。其中,由Noisy-Or和Noisy-And节点组成的贝叶斯网络模型只需要线性数量个参数,与普通贝叶斯网络相比能够进行更高效率的学习和推断,同时也有利于学习到更精确的网络模型;而动态贝叶斯网络用途十分广泛,如跟踪移动机器人、监测高速公路、控制无人驾驶车辆、预测系统的演化状态以及在一定程度上辅助决策等等。广为使用的隐藏马尔可夫模型和卡尔曼滤波器都可以看作是动态贝叶斯网络的特殊形式,可以统一到动态贝叶斯网络的范畴来研究和讨论。 针对当前贝叶斯网络方法的研究难点和研究热点,论文主要进行了以下几个方面的研究工作:(1)提出了一种基于进化计算的贝叶斯网络结构学习方法——EM-EA方法。EM-EA方法将EM算法和进化算法(EA)有机地结合起来,利用EM算法处理不完整数据,利用进化算法学习贝叶斯网络的结构。此外,通过引入新变异算子和对交叉算子进行扩展,EM-EA方法能够学习和进化带有隐藏变量的贝叶斯网络。实验结果表明,EM-EA方法与W. Myers等人的进化方法相比具有更高的效率和精度;与Friedman的SEM方法相比,EM-EA方法与之在学习隐藏变量方面性能相当,但能够在学习的过程中根据需要添加隐藏变量,比SEM更灵活,也更有实用性。 (2)提出了一种贝叶斯网络结构的增量学习方法——IEMA。IEMA将EM-EA方法引入到了贝叶斯网络结构的增量学习过程中,有效克服了增量爬山算法存在的局部极值问题。此外,IEMA改进了Friedman等人的增量学习过程,克服了其在选择参数时的矛盾。试验结果证明了IEMA的有效性。就存储开销而言,IEMA与Friedman等人的增量学习方法相当,但IEMA学到的网络更精确。(3)研究了由Noisy-Or和Noisy-And节点组成的贝叶斯网络的参数学习和结构学习算法。参数学习算法借鉴人工神经网络的误差传播算法,对网络参数的信念和误差进行传播和修正。结构学习算法利用数据中的信息引导搜索有用的网络修正点,必要时能够发现隐藏变量,并能学习带隐藏变量的贝叶斯网络。论文方法能够很方便地用于理论求精,实验表明其性能与现有其它的混合理论求精系统不相上下,但具有更精确的语义,更容易理解。(4)研究了动态贝叶斯网络(DBN:Dynamic Bayesian Networks)的学习和近似推断技术。在DBN学习方面,研究了如何从完整数据集和不完整数据集以及存在隐藏变量时学习DBN的结构。在DBN推断方面,分析了直接应用标准推断技术进行DBN推断的困难所在,研究了似然加权算法的两种改进——ER算法和SOF算法,并将两种算法组合起来,进行了实验分析。实验结果表明这些算法能够有效克服似然加权算法的缺点,特别是ER和SOF的组合算法表现出了非常好的性能。

A Bayesian network (BN) is a kind of tool for data analysis and decision-making based on the statistics and probabilistic theory. It has received more and more concerns of the researchers and become one of the main approaches dealing with the uncertainty in the field of artificial intelligence because of its stable theoretic basis, natural knowledge representation, flexible reasoning ability, as well as convenient decision-making mechanism. At present, a Bayesian network has widely used in many fields, such as data mining, speech recognition, industrial control, economic forecasting as well as medical diagnosis and has wonderful research prospect.In recent years, there has been a growing interest in the problem of learning Bayesian network structures from data. At present, there have been effective methods for learning network structures from complete data. However, it is quite not the case for learning network structures from incomplete data. Furthermore, it is particularly a difficult problem to learn network structures with hidden variables.Another heat point on Bayesian network research is application-oriented research for the special Bayesian networks, which include object-oriented Bayesian networks, tree augmented Bayesian networks, Multiple Sectioned Bayesian Networks, Bayesian networks consisting of Noisy-Or and Noisy-And nodes, Dynamic Bayesian networks (DBNs) and so on. Where Bayesian networks consisting of Noisy-Or and Noisy-And nodes need only linear number of parameters. Compared with general Bayesian networks, more efficient learning and inference can be achieved above it and more accurate network model can be obtained. While DBNs serves a number of purposes and can be used for tracking moving robots, freeway surveillance, controlling an autonomous vehicle, predicting the future evolution of the observed system as well as approximating rational decision-making with a limited horizon. The widely used hidden Markov models (HMMs) and Kalman filters can be treated as the special form of DBNs and be studied and discussed in the same category. As far as the difficulties and heat points on Bayesian network research is concerned, the paper carried out the following research:(1) Putting forward a new method based on evolutionary computing for learning Bayesian network structures from incomplete data——EM-EA. EM-EA combines the EM algorithm with an evolutionary algorithm (EA), transforms the incomplete data to complete data using EM algorithm and then evolves network structures using evolutionary algorithms with the complete data. Through introducing a new mutation operator and expanding the function of the crossover, EM-EA can learn Bayesian networks with hidden variables. The results of the experiments show that EM-EA is more accurate and practical than other network structure learning algorithms that deal with the incomplete data. The results of the experiments show that compared with the EA of W. Myers et al, EM-EA is more accurate and efficient. And in terms of learning network structures with hidden variables, EM-EA is comparable with SEM algorithm of Friedman et al. While EM-EA could create hidden variables on as-needed basis during the learning process, and so is more flexible and practical than SEM algorithm.(2) Putting forward an incremental method for learning Bayesian network structures——IEMA. IEMA introduces the EM-EA algorithm into the incremental learning process of the Bayesian networks, and overcomes the problem that the incremental hill climbing algorithm tends to get into a local maximum. Furthermore, IEMA improves incremental learning process by Friedman et al., and solves the contradiction when selecting the value of the parameter K. The results of the experiments verified the validity of IEMA. In terms of storage cost, IEMA is comparable with the incremental learning method by Friedman et al, while it is more accurate.(3) Studying the algorithms for learning the parameters and structures of the Bayesian networks which consist of Noisy-Or and Noisy-And nodes. The parameter learning algorithm transmits and revises the error of the parameters using the error transmission algorithm of Artificial Neural Networks for reference. The structure learning algorithm uses the information in the data to guide the search for useful revisions, and can discover hidden variables and learn networks with hidden variables when necessary. Furthermore, the method described above can be used for theory refinement. The experiments demonstrate that its performance is comparable to that of other existing hybrid theory refinement systems, while the networks produced by it have more precise semantics and are more comprehensible.(4) Studying the structure learning and approximate inference techniques of DBNs. As for the structure learning of DBNs, the paper introduces EM-EA algorithm into the learning process of DBNs and gives out the flow of learning DBNs in the presence of incomplete data and hidden variables. As for the DBNs inference, the paper analyzes the difficulties in applying directly the standard inference techniques to DBNs, studies two improved algorithms of Likelihood Weighting (LW) algorithm——ER algorithm and SOF algorithm, and assembles the two algorithms. The experimental results show that these algorithms can overcome effectively the shortcomings of LW algorithm, especially the assembled algorithm of ER algorithm and SOF algorithm shows outstanding performance.