Time Series Meta-Learning: Can Zero-shot transfer learning be applied to Time Series Forecasting?

[StatMixedML]

Dear community,

I came across this paper Meta-learning framework with applications to zero-shot time-series forecasting by Yoshua Bengio and colleagues.

They raise the following very interesting question:

Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets?

Using a variety of models and datasets, their conclusion is: Yes we can! Their results are suggesting that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, resulting in performance that is at least as good as that of state-of-practice univariate forecasting models. Mentioning the above paper, we also need to refer to Section 6 in the N-BEATS Paper, where the authors already make a connection between N-BEATS and meta-learning.

The idea is as follows:

Zero-shot TS forecasting task definition: One of the base datasets, a source dataset, is used to train a machine learning model. The trained model then forecasts a TS in a target dataset. The source and the target datasets are distinct: they do not contain TS whose values are linear transformations of each other. The forecasted TS is split into two non-overlapping pieces: the history, and the test. The history is used as model input and the test is used to compute the forecast error metric. The authors define the term zero-shot in the sense that the number of history samples available for the target time series is so small that it makes training a deep learning model on this time series infeasible.

Their results suggest that N-BEATS consistently outperforms most statistical models tailored to the datasets, as well as DeepAR trained on M4 and evaluated in zero-shot regime on other datasets. Also, the authors show that the concept works effectively across multiple datasets and across multiple forecasting tasks (e.g. forecasting yearly, monthly or hourly TS) and even across very loosely related tasks (for example, forecasting hourly electricity demand after training on a monthly economic data after appropriate time scale normalization). The authors also show that N-BEATS can be cast as a Meta-learning Algorithm which is why N-BEATS performs better than DeepAR.

I am very much looking forward to hearing your opinion on the paper and the implications it has on forecasting in general.