心智与世界之间的关系是一个的古老的话题，而语言则是连接二者的桥梁。20世纪哲学的语言转向使人们更加认识到语言的重要性。计算机科学与人工智能也在同时诞生和发展，产生了基于自然语言的图灵测试。时至今日，一个核心问题就是如何对语言进行建模和计算，这对深入了解心智和实现通用人工智能极为重要。 两个与之最为相关的研究方向，是自然语言处理和多智能体可交流强化学习。尽管前者在实际应用中率先落地，我们认为后者更接近语言的本质，并有潜力对各种类型的语言进行建模，而不仅仅针对自然语言。因此本文在兼顾前者的同时，更加强调后者。本文主要涉及到三个关于合作式多智能体语言交流系统的工作，并使用复杂系统科学中的概念将它们串联起来。概括地说，语言系统作为一个宏观整体，是从每个微观智能体间的互动和合作中涌现出来的。我们的工作在提高了语言的效率和适应性的同时，也探索了随之出现的语言现象。 在第一项工作中，智能体之间相互传递消息，以协调动作选择，获得全局最优的价值。以前工作中消息传递属于比较纯粹的分布式限制优化算法，存在性能瓶颈，为了使整个系统得到更高的效率，我们设计了一个中心式的非线性混合网络来整合局部价值，并在分布式通信算法之上增添了一个全局优化算法。这相当于给语言系统在宏观尺度上分配了更多的复杂性。 在第二项工作中，智能体之间通信获取共识信息，以实现它们的共同目标并应对可能的攻击。传统上，为了实现共识，人们会手动设计一个全局的共识协议。然而，这些协议难以灵活适应对攻击或环境设置假设的微小改变。我们的方法是让智能体通信策略的参数可调，以适应这些改变。整个通信系统因此变得更加鲁棒。这相当于在系统的微观尺度上提供更多的复杂性和自由度。 第三项工作是最重要的。当解决指代游戏问题时，一个符号化的语言系统涌现出来。在这里，我们考虑了不同系统尺度之间的跃迁，即微观智能体可以虚拟内嵌宏观世界，其中包括语言系统本身和其他智能体的想法。涌现和跃迁的结合，对获取更自然、准确、强大、细粒度和简洁的语言表达很重要。 我们还将各种多智能体系统迁移到了现实场景，并将人类纳入到和机器智能体的交互中。在大语言模型时代，一个有前景的方向是将跃迁的思想融入到大语言模型的智能涌现里，这可能会指向一些终极问题，如机器意识和心智理论。
The relationship between the mind and the world has been an eternal topic, with language serving as the bridge between the two. The linguistic turn in 20th-century philosophy further highlighted the importance of language. Concurrently, the field of computer science and artificial intelligence started development, along with the famous Turing test, a natural language-based criterion for determining a machine‘s intelligence and mind ability. Needless to say, a central question that concerns our generation is how to model and compute language, which is the most essential way to know more about the mind and achieve Artificial General Intelligence (AGI). This thesis is most related to two lines of research, namely, Natural Language Processing (NLP) and Communication in Multi-agent Communicative Reinforcement Learning (MACRL). While the former field has taken a step ahead in practical applications, we believe that the latter is closer to the essence of language and capable of modeling various kinds of languages, instead of only the natural language, and therefore we place more emphasis on the latter in our research. Using the concepts of Complex Systems Science (CSS), we could concisely organize the idea of our three main works on different cooperative multi-agent language systems. Generally speaking, language systems, as a macroscopic whole, emerge from the microscopic interactions and cooperation of the intelligent agents, and while enhancing their efficiency and adaptability, we inspect the accompanying language phenomena. In the first work, message-passing between the agents assists the coordination of their actions, for an optimized global value. Classical message-passing mechanisms are relatively pure decentralized algorithms, which is limited in efficiency. To achieve higher efficiency on the whole system level, we design a centralized non-linear mixer network to integrate the local values and a global optimization algorithm on top of the local message-passing mechanism. This amounts to arranging more complexity to the macroscopic scale of the language system. In the second work, agents convey consensus information to achieve common goals and deal with possible attacks. Traditionally, people manually design macroscopic consensus protocols to achieve this. However, these protocols are not adaptive enough when facing small changes in the assumption of attack or the environment setting. Our approach is letting the agents adapt their parameters and strategies to the environment. The whole communication system then becomes robust. This amounts to arranging more complexity and freedom to the microscopic scale. The third work is the most important one. A macroscopic symbolic linguistic system emerges from microscopic multi-agent interactions when solving referential games. Then we consider the transition between different system levels, i.e. agents virtually model the world, including the language system itself and how other agents think. The combination of the system emergence and transition manifests its importance for making inroads toward more natural, accurate, robust, fine-grained, and succinct utterances. For application scenarios, we have developed systems for transferring communication-based agent models into more realistic environments, considering humans in the loop. A promising future direction is integrating the idea of transition into the emergence of Large Language Model (LLM) intelligence, pointing to eventual problems like machine consciousness and theory of mind.
