Temporal Distribution Characterization

Temporal Distribution Characterization, or TDC, is a module used in the AdaRNN architecture to characterize the distributional information in a time series.

Based on the principle of maximum entropy, maximizing the utilization of shared knowledge underlying a times series under temporal covariate shift can be done by finding periods which are most dissimilar to each other, which is also considered as the worst case of temporal covariate shift since the cross-period distributions are the most diverse. TDC achieves this goal for splitting the time-series by solving an optimization problem whose objective can be formulated as:

$$ \max _{0<K \leq K_{0}} \max _{n_{1}, \cdots, n_{K}} \frac{1}{K} \sum_{1 \leq i \neq j \leq K} d\left(\mathcal{D}_{i}, \mathcal{D}_{j}\right) $$

$$ \text { s.t. } \forall i, \Delta_{1}<\left|\mathcal{D}_{i}\right|<\Delta_{2} ; \sum_{i}\left|\mathcal{D}_{i}\right|=n $$

where $d$ is a distance metric, $\Delta_{1}$ and $\Delta_{2}$ are predefined parameters to avoid trivial solutions (e.g., very small values or very large values may fail to capture the distribution information), and $K_{0}$ is the hyperparameter to avoid over-splitting. The metric $d(\cdot, \cdot)$ above can be any distance function, e.g., Euclidean or Editing distance, or some distribution-based distance / divergence, like MMD [14] and KL-divergence.

The learning goal of the optimization problem (1) is to maximize the averaged period-wise distribution distances by searching $K$ and the corresponding periods so that the distributions of each period are as diverse as possible and the learned prediction model has better a more generalization ability.