当下，深度神经网络的研究在众多任务中均取得了令人瞩目的成果。但是在面对持续学习的场景时，它们的性能往往会直线下降。而人类日常学习与此不同———即使是婴儿也可以在不知不觉中形成永久记忆。人类的这种能力需要归功于他们的学习方式和大脑树突棘生长机制。了解这一原因将大大有助于研究者们设计出让机器像人类一样持续学习而不产生遗忘的方法。受大脑学习理论的启发，本文针对计算机视觉中持续学习的三个不同问题设置——类别增量分类（CIC）、域增量分类（DIC）和域增量目标检测（DIOD）——各自提出了三种不同的框架，用以解决以上三个问题。首先，针对类别增量分类的特殊问题情景，本文提出了一种基于单个类别样本异常程度度量的持续学习方法（SAME）。该方法不需要存储旧任务样本。对于每一个待学习的类别，该方法会单独训练一个微型模型以防止类别之间的参数覆盖。另外，一个三重交叉熵损失被用于取代传统的分类交叉熵。同时，本方法中加入了外来辅助数据集，辅助数据可以帮助本方法显著增强记忆。在不同数据集上的测试结果表明，SAME具有性能好、鲁棒性强的特点。域增量分类的问题设置中，未来任务的数据分布可能与当下任务不同。本文提出了一种基于合并已学习与正学习的经验的持续学习方法（CLUE）。CLUE通过惩罚特征提取器的异常失真和理想情况下的样本输出的改变来巩固模型对以前和现在任务的记忆。通过实验可以观察到，CLUE实现了显著的性能提升。更加值得一提的是，即使只存储较少的回放样本，CLUE也能保持稳健性，并具有较强的可解释性。对于域增量目标检测，本文提出了旨在节省存储空间的迁移与适应（META）的持续学习框架，该框架包含成对适配器和一个轻量级域预测网络。模型的一部分将被固定权重并直接应用于新任务的新域。而文中采用的适配器用于更新其他部分，它首先适应背景，然后适应细节。另外，域预测网络有助于识别所有已经学习过的域。它通过对存储的特征或下采样后的图像的学习来选择相应的适配器。实验证明META具有较强性能，在避免了遗忘的同时又节省了存储空间。
Even though deep neural networks achieve impressive results on various problems, the performance tends to plummet facing a continuous learning scenario. However, even infants can form a permanent memory imperceptibly, thanks to their learning habits and brain spine growing mechanisms. Allowing machines to learn like humans without forgetting has far-reaching influence. Inspired by brain learning theories, this thesis proposes three learning frameworks for three different problem settings of continual learning in computer vision - Class-Incremental Classification (CIC), Domain-Incremental Classification (DIC), and Domain-Incremental Object Detection (DIOD).A non-replay method named Sample Abnormal Measuring for Each class (SAME) is first proposed for CIC problems, where separated tiny models for classes are trained individually to prevent parameter overwriting, with the addition of a Triple Cross Entropy (TCE) loss that takes advantage of outlier auxiliary datasets to enhance the memory of classes. Results on diverse datasets show that SAME achieves state-of-the-art performances with strong robustness and generalization ability. Concentrating on DIC, where new domains would appear in future tasks, a novel method named Consolidating Learned and Undergoing Experience (CLUE) is proposed. CLUE consolidates former and current experiences by penalizing distortion of feature extractor and alteration of sample outputs. It is observed through extensive experiments that CLUE achieves significant performance improvement, remains robust even with fewer replay samples, and possesses strong explainability.Regarding DIOD, a Memory-Efficient Transferring and Adapting method (META) with pairs of adaptors and a lightweight Domain-Pred Net is proposed. Part of the model will be fixed and transferred to novel domains. Adaptors are used to update other parts, first adapting to the background and then details in the unfamiliar domain. The Domain-Pred Net helps to identify all learned domains and choose corresponding adaptors by learning from stored features or downsampled images. Extensive experiments confirm the advanced performance of META, which both avoids forgetting and saves storage.
