构建达到和超越人类水平的通用人工智能（AGI）是人工智能领域的终极目标。近年来，随着 GPT-4 等大语言模型（LLMs）技术的革命性突破，实现 AGI 的曙光已然初现。然而，LLMs 在复杂推理问题上仍面临瓶颈，尤其是开源模型的推理能力与人类水平相比存在明显差距。这主要表现在大模型存在不稳定行为，例如生成虚构事实或输出有瑕疵的代码，缺乏与外界交互反馈的有效方法。围绕这些大模型的推理问题，本课题分别进行了三个方面的探索，主要内容如下： 首先，揭示大模型自我验证的不可靠性，提出工具化验证框架。 本研究首次系统性论证了LLMs在缺乏外部反馈时自我校正的局限性：实验表明仅依靠自我修正无法有效提升GPT-4等先进模型的性能。通过不确定性估计理论分析，发现LLMs难以准确评估自身知识边界（生成-辨别-批判差距）。为此提出基于交互反思的自我验证框架，在问答任务中的置信度校准准确率超越所有先前方法，并通过工具交互构建验证-修正闭环，在常识推理任务中将事实准确性提升至73\%-80\%。 其次，以交互反思迭代推理，构建 CRITIC 自我改进框架。 受人类批判性思维启发，该框架通过自然语言接口实现LLMs与搜索引擎、代码解释器等工具的深度协作。在三个维度验证其有效性：(1) 在问答任务中让 ChatGPT的F1值提升7.7\%，(2) 在数学程序合成准确率提高最多 16\% 绝对增益，(3) 在毒性内容生成缓解与最先进有监督方法相当，表明了工具交互对持续改进的关键作用。 最后，以工具交互优化智能体，创建工具集成推理新范式，针对数学推理难题，提出融合自然语言推理与符号计算工具的协同架构。通过专家标注的交互轨迹训练和拒绝采样修正策略，在MATH等十个基准测试中实现突破：工具集成推理智能体-7B超越奇才数学-70B达22\%，工具集成推理智能体-34B成为首个突破50\%准确阈值的开源模型，较GPT-4纯思维链推理提升9.8\%，取得了开源模型推理能力的一个里程碑。 本研究通过系统性的理论分析和实证研究，建立了大模型推理能力提升的三位一体方法论：在现象层面揭示自我验证的固有局限，在方法层面构建工具增强的新型框架，在应用层面实现数学推理的实质性突破。系列实验覆盖GPT-3.5, GPT-4、到LLaMA-2等不同规模的开源闭源模型，涉及问答、编程、数学等多元场景，为构建可信赖的AGI推理系统提供了理论依据和技术路径。
Building Artificial General Intelligence (AGI) that matches and surpasses human capabilities remains the ultimate goal in artificial intelligence. Recent revolutionary breakthroughs in Large Language Models (LLMs), exemplified by GPT-4, have revealed the dawn of AGI realization. However, LLMs still face critical bottlenecks in complex reasoning tasks, with open-source models particularly exhibiting significant gaps compared to human-level performance. Key limitations include unstable behaviors such as generating fictional facts or flawed code outputs, along with inadequate methods for interactive external feedback. This research systematically investigates three key directions to address these reasoning challenges: First, revealing the unreliability of self-verification in LLMs and proposing a tool-based verification framework. This work presents the first systematic demonstration of LLMs' limitations in self-correction without external feedback: experiments show that self-revision alone fails to improve performance even in advanced models like GPT-4. Through uncertainty estimation theory analysis, we identify LLMs' inherent difficulty in accurately assessing their knowledge boundaries (Generation-Identification-Critique Gap). We propose an interaction-based self-verification framework that achieves state-of-the-art confidence calibration accuracy in QA tasks, and constructs a verification-correction loop through tool interaction, improving factual accuracy to 73\%--80\% in commonsense reasoning. Second, developing the \critic{} self-improvement framework through iterative reflective reasoning. Inspired by human critical thinking, this framework enables deep collaboration between LLMs and tools (e.g., search engines, code interpreters) via natural language interfaces. Comprehensive validation demonstrates: (1) 7.7\% F1 score improvement for ChatGPT in QA tasks, (2) up to 16\% absolute accuracy gain in mathematical program synthesis, and (3) toxicity mitigation comparable with state-of-the-art supervised methods, highlighting the crucial role of tool interaction for continuous improvement. Finally, creating a novel tool-integrated reasoning paradigm through agent optimization. For mathematical reasoning challenges, we propose a synergistic architecture combining natural language reasoning with symbolic computation tools. Through expert-annotated interaction trajectory training and rejection sampling refinement, we achieve breakthroughs across ten benchmarks including MATH: our 7B model surpasses WizardMath-70B by 22\%, while our 34B becomes the first open-source model exceeding the 50\% accuracy threshold, outperforming GPT-4's pure chain-of-thought reasoning by 9.8\%, marking a milestone in open-source model reasoning capabilities. Through systematic theoretical analysis and empirical studies, this research establishes a trinity methodology for enhancing LLM reasoning: revealing fundamental limitations of self-verification at the phenomenological level, constructing tool-augmented frameworks at the methodological level, and achieving substantial breakthroughs in mathematical reasoning at the application level. Extensive experiments cover various-scale models from GPT-3.5/GPT-4 to LLaMA-2, spanning diverse scenarios including QA, programming, and mathematics. This work provides both theoretical foundations and technical pathways for developing trustworthy AGI reasoning systems.
