舌诊是中医的常用诊断方法，具有简单实用、非侵入式等优点。数字化舌诊是指通过数字图像处理等方法进行舌象分析的技术。现有的数字化舌诊仪器大都只适用于封闭或半封闭环境，应用场景有限，且拍摄时需要接触患者，存在交叉感染风险。此外，它们对舌象的处理和分析结果精度不够高，无法满足实际应用的要求。这些问题都使得舌诊无法进一步推广并服务于更多受众。为此，论文研究一种面向开放环境的数字化舌诊系统，该系统以手机为终端采集舌象并显示舌象处理结果。论文对该系统中的关键算法进行了研究，具体包括：针对有无充足训练数据两种场景，论文提出了两种舌象分割算法。一种基于阈值法和水平集算法，无需训练数据。论文将两种经典的水平集算法相结合，改进了权重函数，使得两种模型在分割中的权重更加均衡。实验表明该算法的分割准确率优于其他传统图像分割算法。另一种基于UNet++深度学习网络，需要一定量的训练数据。论文通过数据增强、迁移学习、修改编码器结构和后处理，使得模型在舌象图片上的分割精度进一步提高。实验表明，该算法的分割失败率仅为2.32%，可以满足实际应用的要求。针对开放环境中不同光照引起的颜色不统一和失真问题，论文提出一种“离线训练，在线校正”的舌象颜色校正算法。该方法使用从标准色卡中筛选出的色块，训练全连接神经网络，并根据环境色温和色卡色差将开放环境归并为9种典型环境。用户仅需从中选择与拍摄环境最接近的一个即可完成校正，无需使用色卡。实验结果显示，该算法在部分环境下可以取得较好的校正效果，其中在白色LED灯照明环境下的平均色差最小，为7.37。针对舌象特征识别中普遍存在的数据集标签不平衡问题，论文将自监督学习与重采样、重加权方法相结合进行处理。先使用无标签数据训练代理任务，再使用有标签数据对模型的全连接层进行微调，并加入重采样和重加权方法减小不平衡问题的影响。实验表明，该方法可以在一定程度上解决不平衡导致的小样本类别识别精度差的问题。在舌苔厚薄分类中，样本数较少的厚和样本数较多的薄都能达到89%的召回率。在舌苔润燥分类中，相较于原始的ResNet-34模型，样本数较少的燥的召回率可以提高30%。
Tongue diagnosis is a common diagnosis method in traditional Chinese medicine, which is easy, useful and non-invasive. Digital tongue diagnosis is the technology which analyzes tongue images using methods like digital image processing. Most existing digital tongue diagnosis devices are only applicable to closed or semi-closed environments, which limits their applications. Patients have to contact with the device when being photographed, which may lead to cross infection. Besides, the analysis results of tongue images are not accurate enough to meet the requirement for practical application. Tongue diagnosis can’t be further promoted and utilized by more people due to the above problems. Aimed at solving these problems, a digital tongue diagnosis system applied in open environments is studied in this paper, which uses cell phones as the terminal to capture tongue images and display analysis results. The key algorithms in this system are studied in this paper, which includes:Depending on whether enough training data is available, two tongue segmentation methods are put forward. The first one is based on thresholding and level set methods, which requires no training data. Two classic level set methods are combined by an improved weight function, which makes the weights of two methods more balanced during segmentation. Experiment results show the accuracy of this method is higher than other classic image segmentation methods. The other one is based on a deep learning network UNet++, which requires certain amount of training data. By transfer learning, data augmentation, changing the encoder structure and post processing, the segmentation results on tongue images can be more accurate. Experiments show there is only a 2.32% failure rate, which can meet the requirements for practical use. A tongue color correction method based on “offline training, online correction” is proposed to solve the color discrepancy and distortion problem caused by different illuminations in open environments. Color patches are selected from a standard color checker to train a fully connected neural network. Open environments are divided and merged into 9 typical environments based on the color temperature and color difference of checkers. The correction can be finished without a color checker once the closest environment to current environment is selected by the user. Experiment results indicate that this method can obtain good correction results in some environments, among which, the environment illuminated by white LED lights has the smallest average color difference of 7.37.Self-supervised learning is combined with re-sampling and re-weighting methods to deal with label imbalance problem which is not uncommon in tongue feature identification tasks. Unlabeled data are used to train a pretext task before the fine- tuning of fully connected layers using labelled data. Re-sampling and re-weighting methods can also be used during fine-tuning to reduce the influence of imbalanced data. Experiment results show that the poor accuracy on minority classes caused by data imbalance can be improved to some extent using this method. In the thickness classification task of tongue coating, both the minority class “thick” and the majority class “thin” can reach a recall of 89%. In the wetness classification task of tongue coating, the recall on minority class “dry” can be improved by 30% compared with original ResNet-34.
