时间序列是现实世界中常见的数据，它由一系列时间戳下记录的观测值构成。时间序列预测是一类重要的任务，它使用过去一段时间的历史观测值预测其未来的变化规律，在气象预报、金融市场等领域有广泛的应用。同一时刻的时间序列数据可以包括多个变量的观测值，例如气象数据可以包括温度、湿度、气压等变量。这些变量携带的信息在时间变化中相互关联，给预测任务带来了额外的信息；与此同时，如何对多变量时序进行建模、合理利用这部分信息，则给时间序列预测带来了挑战。本文即关注多变量时间序列的预测学习问题。变换器网络（Transformer）是一类经典的序列建模架构，在自然语言处理、图像处理等领域有着广泛的应用，也是当前最常用的时间序列分析模型。然而最近提出的时间线性模型（Linear Model）使用简单的结构达到了与变换器网络类似的预测精度，因此变换器网络在多变量时间序列预测任务上的效果正在受到质疑。本文分析了变换器网络用于多变量时间序列预测时的计算过程，认为其将单一时间点的变量视作令牌（Token）的做法破坏了时间维度的连续语义。基于此，本文提出了倒置变换器网络（iTransformer, Inverted Transformer），将时间序列的时间维度与变量维度倒置，将每一变量下的整条时间序列视作令牌，注意力机制用于建模变量间的相关关系，全连接层用于建模时间特征的依赖关系。这一创新在不改变变换器网络内部结构的情况下，实现了模型的结构与数据固有特征的匹配。本文对倒置变换器网络进行了实验验证。在公开数据集中，倒置变换器网络的预测误差低于当前的主流预测算法，在高维多变量时序数据上优势更为明显。本文验证了拓展倒置变换器网络的有效性，使变换器网络衍生算法得以适配于多变量时间序列预测任务。此外，本文展示了倒置变换器网络的维度拓展性与计算效率。最后，本文测试了倒置变换器网络的实际应用效果。基于与蚂蚁集团合作的项目《面向绿色计算的时序机器学习理论与关键问题研究》，本文将倒置变换器网络应用于服务器流量预测任务，应用测试结果表明倒置变换器网络有效提升了预测准确性。
Time series are a common type of data in the real world, recording the process of variables changing over time. Time series forecasting is an important application in time series analysis tasks, which predicts the future series based on the past lookback windows. It has been widely used in meteorological forecasting and financial markets. Time series data at certain time steps may include multiple variables. For example, meteorological data includes temperature series, humidity series and air pressure series. The information carried by these variables entangles over time, bringing additional information but challenges to time series forecasting. In this work, we especially focus on multivariate time series forecasting tasks.Transformer is a classic type of sequence analysis algorithm and is currently the most popular method for time series analysis. However, researchers are challenging its performance in multivariate time series forecasting tasks. In this paper, we analyze the structure of Transformer for multivariate time series forecasting. We believe that using separate time steps as tokens disrupts the semantics of continuous time series. Based on the above motivations, we believe it is not that Transformer is ineffective for time series forecasting, but rather it is improperly used. Thus, we propose iTransformer, which simply applies the attention and feed-forward network on the inverted dimensions. This innovation matches the structure of the model and the intrinsic features of multivariate time series without changing the internal structure of the Vanilla Transformer.We conduct experiments to show the effectiveness of the proposed iTransformer. In public real-world datasets, iTransformer achieves state-of-the-art, where averaged MSE and MAE are 14.6\% and 11.3\% lower than current SOTA method respectively. The Extended iTransformers further empower the Transformer family with promoted performance, making it a nice alternative as the fundamental backbone of time series forecasting. We also conduct experiments to show the generality and efficiency of iTransformer. Finally, based on collaboration with Ant Group, we utilize iTransformer on server flow forecasting tasks. The results show that iTransformer outperforms the current SOTA method by 16.2% in MSE and 14.7% in MAE respectively.
