Linux内核被广泛应用于各类信息系统，内核的安全漏洞对这些信息系统造成了严重的威胁。因此，对Linux内核中的安全漏洞进行检测与挖掘具有重要意义。Linux内核中丰富的程序状态可以作为知识来帮助检测与挖掘漏洞，然而巨大的状态空间让追踪和使用程序状态都变得十分困难。本文对状态敏感的Linux内核漏洞检测与挖掘展开研究，围绕如何感知状态、如何利用状态判定漏洞、如何利用状态引导漏洞挖掘这三个研究问题提出了解决方案。本文的研究内容和贡献如下：1.基于静态分析的状态自动化识别与监控技术：复杂的程序中包含大量的程序状态，但是状态空间过于巨大导致无法直接对状态表示与追踪。本文对开源项目进行了分析，提出了一个基于变量表征的状态模型，通过追踪表示状态的变量来追踪近似的程序状态。本文总结了表示状态的变量的特征，并提出了基于静态分析的自动化状态变量识别方案，然后通过编译时插桩的方式对状态变量取值进行监控。原型系统SV-Collector在Linux上游内核中识别出6055个状态变量，其中有接近50%的变量名语义与状态有关，漏报率仅为10%。2.基于状态监控Docker容器逃逸漏洞检测技术：Docker容器在云服务中被广泛使用，现有的漏洞检测方案缺乏对Linux内核安全机制相关状态的感知，无法检测容器逃逸漏洞等内核逻辑漏洞。本文对Docker容器实现原理进行分析，研究如何从繁多状态中挑选用于检测逃逸漏洞的关键状态；根据Docker容器逃逸的两个必要条件，研究如何对关键状态进行检查。本文提出了一种基于进程安全状态监控的容器逃逸漏洞检测方案DockerSan，该方案通过在系统调用返回时监控进程中隔离相关的安全状态是否异常来检测逃逸漏洞。DockerSan能够检测的逃逸漏洞数目是现有方案Falco的6倍，并结合模糊测试成功发现了4个已知逃逸漏洞。3.基于状态覆盖反馈的模糊测试技术：现有基于代码覆盖率的模糊测试无法深入地探索程序状态。本文提出一个基于状态覆盖反馈的模糊测试方案StateFuzz挖掘Linux内核中的内存破坏漏洞。该方案通过跟踪相关状态变量的值域范围组合以解决状态空间过大的问题，使用三维反馈机制来引导模糊测试充分探索程序代码和程序状态。StateFuzz在对Linux上游内核驱动和真实手机驱动的模糊测试中，共发现20个未知漏洞并获得了15个CVE编号，其中9个漏洞被授予了漏洞赏金奖励，同时StateFuzz能获得比Syzkaller更高的代码覆盖率和状态覆盖率。

The Linux kernel is widely used in various devices in cyberspace and is the core component of the systems of these devices. Vulnerabilities in the Linux kernel pose a serious threat to these devices and can cause their systems to be controlled or crash. Therefore, the detection and discovery of vulnerabilities in the Linux kernel are of great significance to the security of cyberspace. Program states in the Linux kernel can be used as knowledge to help detect and discover vulnerabilities. However, the huge value space of program states makes it difficult to track and utilize program states. In this thesis, we conduct an in-depth study of the program states of the Linux kernel and propose solutions to three research questions: how to track program states, how to use program states to detect vulnerabilities, and how to use program states to guide vulnerability discovery. The main contributions of this thesis are as follows.1. Automatic State Identification and State Monitoring Technology Based on Static Analysis: Complex programs contain a large number of program states, but the state space is too large to track directly in practice. We analyze open-source projects and propose a state model based on variable representations to track approximate program states by tracking the values of variables that represent program states. We further summarize the characteristics of the variables representing program states, propose an automated state-variable identification solution based on static analysis, and then we can monitor the state-variable by code instrumentation. The prototype system SV-Collector identifies 6055 state-variables in the Linux upstream kernel, nearly 50% of which have a variable name with semantics related to program states, and the false negative rate is only 10%.2. Detection Technology of Docker Container Escape Vulnerability Based on State Monitoring:Docker containers are widely used in cloud services. However, existing vulnerability detection solutions lack awareness of the security mechanism-related states of the Linux kernel and cannot detect kernel logic vulnerabilities such as container escape vulnerabilities. Through analyzing the implementation of the Docker container, this thesis studies how to select the key states for container escape vulnerability detection from various states, and studies how to monitor the key states according to the two necessary conditions for the Docker container escape. This thesis proposes a container escape vulnerability detection solution DockerSan based on security state monitoring. DockerSan detects escape vulnerabilities by checking whether violations exist in the isolation-related states when a system call returns. DockerSan can detect 6 times more vulnerabilities than the existing solution Falco and successfully finds 4 known escape vulnerabilities in combination with fuzzing.3. Fuzzing Technology Based on State Coverage Feedback:Existing code coverage guided fuzzers cannot deeply explore program states. This thesis proposes a state coverage guided fuzzing approach StateFuzz to discover memory corruption vulnerabilities in the Linux kernel. StateFuzz uses a three-dimensional feedback mechanism to guide the fuzzer to deeply explore program code and program states, by tracking the combination of value ranges of relevant state-variables rather than the combination of all state-variables, to avoid state explosion caused by a too-large state space. StateFuzz finds 20 unknown vulnerabilities and obtains 15 CVE IDs in fuzzing drivers of the Linux upstream kernel and a real Android phone, 9 of the confirmed vulnerabilities are assigned with vulnerability bounty rewards. Moreover, StateFuzz can obtain higher code coverage and state coverage than Syzkaller.