Lee, Daesoo; Malacarne, Sara & Aune, Erlend (2023)
Vector Quantized Time Series Generation with a Bidirectional Prior Model
Proceedings of Machine Learning Research (PMLR), 206, s. 7665- 7693.
Time series generation (TSG) studies have mainly focused on the use of Generative Adversarial Networks (GANs) combined with recurrent neural network (RNN) variants. However, the fundamental limitations and challenges of training GANs still remain. In addition, the RNN-family typically has difficulties with temporal consistency between distant timesteps. Motivated by the successes in the image generation (IMG) domain, we propose TimeVQVAE, the first work, to our knowledge, that uses vector quantization (VQ) techniques to address the TSG problem. Moreover, the priors of the discrete latent spaces are learned with bidirectional transformer models that can better capture global temporal consistency. We also propose VQ modeling in a time-frequency domain, separated into low-frequency (LF) and high-frequency (HF). This allows us to retain important characteristics of the time series and, in turn, generate new synthetic signals that are of better quality, with sharper changes in modularity, than its competing TSG methods. Our experimental evaluation is conducted on all datasets from the UCR archive, using well-established metrics in the IMG literature, such as Frechet inception ´ distance and inception scores.
Time series forecasting is an important problem, with many real world applications. Transformer models have been successfully applied to natural language processing tasks, but have received relatively little attention for time series forecasting. Motivated by the differences between classification tasks and forecasting, we propose PI-Transformer, an adaptation of the Transformer architecture designed for time series forecasting, consisting of three parts: First, we propose a novel initialization method called Persistence Initialization, with the goal of increasing training stability of forecasting models by ensuring that the initial outputs of an untrained model are identical to the outputs of a simple baseline model. Second, we use ReZero normalization instead of Layer Normalization, in order to further tackle issues related to training stability. Third, we use Rotary positional encodings to provide a better inductive bias for forecasting. Multiple ablation studies show that the PI-Transformer is more accurate, learns faster, and scales better than regular Transformer models. Finally, PI-Transformer achieves competitive performance on the challenging M4 dataset, both when compared to the current state of the art, and to recently proposed Transformer models for time series forecasting.
Lee, Daesoo; Aune, Erlend, Langet, Nadege & Eidsvik, Jo (2022)
Ensemble and Self-supervised Learning for Improved Classification of Seismic Signals from the Åknes Rockslope
A case study with seismic geophone data from the unstable Åknes rock slope in Norway is considered. This rock slope is monitored because there is a risk of severe flooding if the massive-size rock falls into the fjord. The geophone data is highly valuable because it provides 1000 Hz sampling rates data which are streamed to a web resource for real-time analysis. The focus here is on building a classifier for these data to distinguish different types of microseismic events which are in turn indicative of the various processes occurring on the slope. There are 24 time series from eight 3-component geophone data for about 3500 events in total, and each of the event time series has a length of 16 s. For the classification task, novel machine learning methods such as deep convolutional neural networks are leveraged. Ensemble prediction is used to extract information from all time series, and this is seen to give large improvements compared with doing immediate aggregation of the data. Further, self-supervised learning is evaluated to give added value here, in particular for the case with very limited training data.
Vassøy, Bjørnar; Ruocco, Massimiliano, de Souza da Silva, Eliezer & Aune, Erlend (2019)
Time is of the essence: A joint Hierarchical RNN and Point Process model for time and item predictions
Jung, Jason J. (red.). WIMS2019 Proceedings of the 9th International Conference on Web Intelligence, Mining and Semantics
Aune, Erlend; Eidsvik, Jo & Ursin, Bjørn (2013)
Three-dimensional non-stationary and non-linear isotropic AVA inversion
Geophysical Journal International, 194(2), s. 787- 803. Doi: 10.1093/gji/ggt127
Aune, Erlend; Eidsvik, Jo & Pokern, Y (2013)
Iterative numerical methods for sampling from high dimensional Gaussian distributions