本文研究了强不规则引力场中的拓扑动力系统，包括强不规则天体引力场中质点运动的不同形式的动力学方程与有效势、平面对称引力场中运动的秩序与混沌、强不规则天体引力场中平衡点附近的轨道与流形以及大范围周期轨道、稳定性与流形结构。给出了任意坐标系和特殊坐标系下的包含引力势及有效势两种类型的分量形式的动力学方程，导出了特殊坐标系下有效势和Jacobi积分的简化形式。给出了9种新的形式的动力学方程及有效势等，包括系数矩阵形式、Lagrange形式、Hamilton 形式、辛形式、Poisson形式、Poisson括号形式、上同调形式和Kähler 流形上及第二复流形上的动力学方程。 随后，研究了旋转平面对称引力场中的周期轨道、流形与混沌运动。发现平衡点附近的动力学行为完全由平衡点附近的子流形与子空间的结构所确定。将非退化的平衡点分为12种不同的类型，建立了平衡点线性稳定、非共振的不稳定以及共振的充分必要条件。进一步发现共振平衡点是Hopf分岔点，它将导致共振平衡点附近的混沌运动；发现在参数变化下，共振平衡点附近会出现周期轨道族的产生与消失现象。 进一步，将研究对象扩展为一般的强不规则天体，研究了其平衡点的特性、平衡点附近的轨道与流形。给出了平衡点附近线性化的动力学方程和平衡点的特征方程。提出并证明了平衡点稳定的一个充分条件和一个充分必要条件。提出了一个非黎曼度量，并证明了在该度量下，轨道和测地线等价。使用特征方程的根，将非退化平衡点分为8种类型。 讨论了线性稳定、非共振的不稳定以及共振平衡点的特性及其附近的动力学规律。发现了共振平衡点的共振流形的维数至少是4维的，且共振平衡点附近至少存在一族周期轨道。相关理论结果应用研究小行星216 Kleopatra、1620 Geographos、4769 Castalia 和6489 Golevka的平衡点附近的动力学行为中。此后，研究了大范围轨道及其对应的特征乘子的分布、周期轨道、轨道的稳定性与子流形结构。发现一般强不规则天体引力场中的大范围轨道有34种不同的拓扑类型，包括6种普通情形、3种碰撞情形、3种纯退化实鞍情形、7种纯周期情形、7种纯倍周期情形以及1种周期兼碰撞情形、1种周期兼退化实鞍情形、1种倍周期兼碰撞情形、1种倍周期兼退化实鞍情形和4种周期兼倍周期情形。发现了特征乘子的不同分布决定了子流形的结构、轨道的类型、动力学行为与相图结构等规律。给出了各种情形的相关性质。最后将这一理论结果应用到小行星6489 Golevka 和243 Ida附近的动力学研究中。

This thesis investigates the topological dynamical system in the highly irregular gravity field, including the different novel forms of the dynamical equations of a particle orbiting highly irregular-shaped celestial bodies, order and chaos in a rotating plane-symmetric potential field, orbits and manifolds near the equilibrium points of a rotating highly irregularly celestial body, as well as the global periodic orbits, the stability of orbits and the structure of submanifolds in the potential field of rotating highly irregular-shaped celestial bodies.The different novel forms of the dynamical equations of a particle orbiting a rotating irregular celestial body and the effective potential, the Jacobi integral, etc. on different manifolds are presented. Nine new forms of the dynamical equations of a particle orbiting a rotating irregular celestial body are presented, and the classical form of the dynamical equations has also been found. The dynamical equations with the potential and the effective potential in scalar form in the arbitrary body-fixed frame and the special body-fixed frame are presented and discussed. Moreover, the simplified forms of the effective potential and the Jacobi integral have been derived. The dynamical equation in coefficient-matrix form has been derived. Other forms of the dynamical equations near the asteroid are presented and discussed, including the Lagrange form, the Hamilton form, the symplectic form, the Poisson form, the Poisson-bracket form, the cohomology form, and the dynamical equations on the Kähler manifold and another complex manifold. Periodic orbits, manifolds and chaos in a rotating plane-symmetric potential field are studied; it is found that the dynamical behaviour near the equilibrium point is completely determined by the structure of the submanifolds and subspaces near the equilibrium point. The non-degenerate equilibrium points are classified into twelve cases. The theory developed here is lastly applied to two particular cases, the rotating homogeneous cube and the circular restricted three-body problem.The orbits and manifolds near the equilibrium points of a rotating highly irregularly celestial body are discussed. The linearised equations of motion relative to the equilibrium points in the gravitational field of a rotating body, the characteristic equation and the stable conditions of the equilibrium points are derived and discussed. First, a new metric is presented to link the orbit and the geodesic of the smooth manifold. Then, using the eigenvalues of the characteristic equation, the equilibrium points are classified into 8 cases. A theorem is presented and proved to describe the structure of the submanifold as well as the stable and unstable behaviours of a massless test particle near the equilibrium points. The linearly stable, the non-resonant unstable, and the resonant equilibrium points as well the dynamical laws around them are discussed. As an application of the theory developed here, relevant orbits for the asteroids 216 Kleopatra, 1620 Geographos, 4769 Castalia and 6489 Golevka are studied. Moreover, the distribution of characteristic multipliers, the stability of orbits, periodic orbits and the structure of submanifolds in the potential field of rotating highly irregular-shaped celestial bodies are also studied. The topological structure of submanifolds for the orbits in the potential field of a rotating highly irregular-shaped celestial body is discovered that it can be classified into 34 different cases. It is found that the different distribution of characteristic multipliers fixes the structure of submanifolds, the types of orbits, the dynamical behavior and the phase diagram of the motion. Classifications and properties for each case are presented. The theory developed here is applied for the asteroids 6489 Golevka and 243 Ida to find the dynamical behaviour around these irregular-shaped bodies.