在过去十几年中，由于在非线性光学和 Bose-Einstein 凝聚等物理问题中有着重要的应用，非线性薛定谔方程组得到了广泛的关注。很多著名的数学家对非线性薛定谔方程组做了大量杰出的研究工作。本文主要是运用变分法和椭圆方程的理论来研究某些非线性薛定谔方程组的非平凡解的存在性和相关性质。首先，我们考虑带有立方次幂非线性项的 Bose-Einstein 凝聚型方程组。在次临界情形（即空间维数为 2 或 3），我们研究基态解存在时参数的最优范围、基态解的唯一性和渐近收敛行为。这给出了 Sirakov 在文献 (Comm. Math. Phys., 2007, 271: 199-221) 中提出的一个公开问题的第一个回答。另外，对任意负的耦合系数我们研究了无穷多个变号解的存在性和相关性质。在临界情形（即空间维数为 4），我们系统研究了这个问题的基态解，包括存在性、非存在性、唯一性和耦合系数趋于负无穷时基态解的极限产生的相位分离现象。这似乎是这类方程组在 4 维情形下的第一个结果。另外，我们可以将上面的部分结果推广到一般的临界方程组（即空间维数大于等于 5 时相应的临界方程组）。与 4 维的特殊情形相比，我们发现会有很多不一样的现象发生。例如，我们可以证明对任意非零的耦合系数都存在基态解，但是这在 4 维的特殊情形下不成立。当空间维数大于等于 6 且耦合系数趋于负无穷时，我们可以证明方程组的基态解收敛到 Brezis-Nirenberg 临界指数问题的变号解。所以这个一般的临界方程组与著名的 Brezis-Nirenberg 临界指数问题密切相关。这里我们研究了开球上 Brezis- Nirenberg 临界指数问题最低能量解的唯一性和最优能量估计，这在上面临界方程组的研究中很有用。同时我们也证明了一般有界光滑区域上 Brezis-Nirenberg 临界指数问题多解的存在性。最后，我们考虑带有临界指数的 Ambrosetti 型线性耦合的薛定谔方程组。我们对不同范围的耦合系数研究了基态解的存在性与非存在性。注意我们的结果是几乎最优的。
In the last decades, nonlinear Schrodinger systems have received a lot of attention,since they have great applications to many physical problems such as nonlinear optics and Bose-Einstein condensation. Many famous mathematicians have made a lot of excellent researches on nonlinear Schrodinger systems.In this thesis, we study the existence and qualitative properties of nontrivial solutions for some nonlinear Schrodinger systems via variational methods and elliptic equations theories. Firstly, we consider a Bose-Einstein condensation systemwith cubic nonlinearities. In the subcritical case (i.e. the spatial dimension is 2 or 3), we study the optimal parameter range for the existence of ground state solutions, the uniqueness and asymptotic behaviors of ground state solutions. These give the first partial answer to an open question raised by Sirakov in (Comm. Math. Phys., 2007, 271: 199-221). We also obtain the existence and related properties of infinitely many sign-changing solutions for any negative coupling constants. In the critical case (i.e. the spatial dimension is 4), we make a systematical research on ground state solutions of this problem, including the existence, the nonexistence, the uniqueness and the phase separation phenomena of the limit profile as the coupling constant tending to minus infinity. These seems to be first results for this system in the dimension 4 case. Besides, we can extend some results above to a general criticalsystem (i.e. a homologous critical problem with spatial dimensions at least 5). It turns out that some quite different phenomenon happen comparing with the dimension 4 case .For example, we can prove the existence of ground state solutions for any nonzero coupling constant, which can not hold in the dimension 4 case. When the spatial dimensions are at least 6 and the coupling constant tends to minus infinity, we can prove that the ground state solutions of this system converge to sign-changing solutions of the Brezis-Nirenberg critical exponent problem. Hence this general critical system is related closely to the well-known Brezis-Nirenberg critical exponent problem. Here we study the uniqueness and sharp energy estimates of least energy solutions for the Brezis-Nirenberg critical exponent problem in a ball, which is very useful in the study of the above general critical system. We also obtain multiple solutions for the Brezis-Nirenberg critical exponent problem in a general smooth bounded domain. Finally, we consider Ambrosetti type linearly coupled Schrodinger equations with critical exponent. We study the existence and nonexistence of ground state solutions for different coupling constants. Remark that our result is almost optimal.
