材料的各向异性电磁性质使电磁波在其中传播时产生很多奇异的现象，其中，由非正定介质的强各向异性产生的全角负折射、传导倏逝波等超常电磁特性，具有重要的研究价值与应用前景，近年来受到国内外研究者的广泛关注。目前，非正定介质的研究主要集中在具有非正定介电张量的超材料及其负折射行为，其应用研究主要是实现具有亚波长分辨率的超透镜成像。本论文主要通过对具有层状结构晶体的介电性能的研究、对磁谐振单元阵列材料的磁导率的研究，提出了基于自然晶体中本征介电性能的非正定电介质以及基于超材料的非正定磁介质，深入研究了这些材料的超常电磁行为，设计了三种基于非正定介质超常电磁特性的新型光学器件。本论文的主要创新点如下： 首次在自然材料中得到电磁波的全角负折射现象。首先，通过椭圆偏振测量术研究石墨单晶、MgB2单晶的介电常数的非正定性，发现了石墨在紫外频段、MgB2单晶在蓝紫光频段的负折射行为。其次，通过研究铜基高温超导材料以及Ruddlesden-Popper相锶钌氧材料等具有层状晶体结构的化合物的反射谱发现这些材料在某些特定频段具有非正定介电张量，分析了这些材料的非正定介电常数并预测了可以发生负折射的频段。最后，对目前已发现的自然非正定介质的机理进行了总结与分类。 提出了一种基于椭圆偏振测量术来验证材料折射性质的测试方法。与传统的透射测试法相比，这种方法避免了光频段测试尺度的限制，简单易行。文中以此方法验证了石墨与MgB2单晶材料的负折射行为。 提出了一种基于材料非正定磁导率的低损耗负折射现象。制备了具有非正定磁导率的密排SRR阵列超材料并验证了该超材料的低损耗负折射行为；通过分析测试结果研究了该超材料反射、透射性能，并利用电磁学经典理论研究了密排SRR环之间的耦合作用，解释了密排SRR阵列超材料低损耗的原因。 提出了利用非正定介质的强各向异性改善基于双折射晶体的传统光学器件的性能的思路，设计了一种基于非正定介质的具有大分束角的高性能平行偏振光分束器。此外，提出了利用具有极强各向异性的非正定介质控制电磁波传播的方法，与目前基于变换光学设计的超材料实现电磁波控制的方法相比，非正定介质结构简单，质地均匀，工业上更容易实现。

The anisotropic electromagnetic (EM) property of the materials usually brings in many extraordinary phenomena when the EM wave propagates in these materials. Among of them, indefinite medium, due to its strong anisotropy, can realize an all angle negative refraction of the EM wave and support the propagation of the evanescent waves, which is of significant value for the scientific research and potential applications, and has attracted much attention in the world. So far, the research on the indefinite media mainly focuses on the fabrication of the metamaterial with indefinite dielectric tensor and its negative refraction behavior, with the main application as a hyperlens whose resolution can break through the diffraction limit. In this dissertation, the abnormal EM behaviors in the natural indefinite media with layered crystal structure and metamaterial of indefinite permeability produced by the magnetic resonance unite are studied theoretically and experimentally. Additionally, three novel optical devices are designed based on the indefinite media. The key findings and innovation points of this dissertation are as follows: An all angle negative refraction is realized by natural materials for the first time. Firstly, indefinite permittivity in the single crystalline graphite and MgB2 are studied and all angle negative refractions in these two materials are verified by ellipsometer experimentally. Secondly, by analyzing the reflection spectra of the cuprates and Ruddlesden-Popper materials of Srn+1RunO3n+1, indefinite permittivities are found and the anticipated negative refraction frequency regions are computed in these material. Finally, the natural indefinite mechanism is derived, summarized and classified based on all the known natural indefinite media currently. A new method of refraction behavior exploration is proposed based on the ellipsometry. In contrast to the traditional measurement by transmission, this method avoids the scale constraints in the optical measurement, which is quite easy to manipulate. It has been used to verify the negative refraction behavior in the single crystalline graphite and MgB2. A low loss negative refraction is realized by a metamaterial with an indefinite permeability. A metamaterial of closely arranged SRRs arrays is fabricated and the negative refraction behavior in the metamaterial is observed experimentally. The reflection and transmission properties are analyzed according to the experiment results. Moreover, the coupling effect in the closely arranged SRRs array is derived by the classic electromagnetism, which explains the low loss effect of the metamaterial. The idea of improving the conventional birefringence crystal based optical devices though the strong anisotropy of the indefinite medium is proposed and is used to design an efficient polarization beam splitter with large splitting angle. Additionally, indefinite media with extremely strong anisotropy are used to control the propagation of the electromagnetic wave. Compared with the contemporary metamaterial obtained by the transformation optics, the structure of the indefinite medium is simple and homogenous, which is feasible to realize by the current technology.