Sepformer-based Models: More Efficient Models for Long Sequence Time-Series Forecasting

Forecasting long sequence time series plays a crucial role in many applications such as anomaly detection and financial predictions. Achieving consistently good results requires a model that can precisely capture the long-range dependencies in input sequences. And very few current models can meet the requirements. Informer has recently demonstrated state-of-the-art accuracy in LSTF. Yet several other aspects of its performance leave much room for improvement. These include: 1) complexity - Informer has a relatively high computational complexity and a high memory overhead; 2) nuance - there is limited ability to capture the subtle features in a data stream; 3) interpretability - the inference procedure of Informer is not explainable; 4) extensibility - accuracy is poor with extra-long multivariate time series. To address these issues, we propose a suite of models under the banner Sepformer. The set comprises Sepformer and two variants SWformer and Mini-SWformer. Sepformer uses separate networks to extract data stream features in parallel. SWformer and Mini-SWformer dramatically separate high-frequency and low-frequency components to process the data stream and reduce the requirement for GPU memory by adopting a discrete wavelet transform. Extensive experiments show that the Sepformer models substantially outperform state-of-the-art methods in terms of accuracy, computational complexity and usage of GPU memory use.