Sparsity exploitation via discovering graphical models in multi-variate time-series forecasting

1 code implementation • 29 Jun 2023 • Ngoc-Dung Do, Truong Son Hy, Duy Khuong Nguyen

Second, we fit these graph structures and the input data into a Graph Convolutional Recurrent Network (GCRN) to train a forecasting model.

Time Series Time Series Forecasting

Improving Transformers with Probabilistic Attention Keys

1 code implementation • 16 Oct 2021 • Tam Nguyen, Tan M. Nguyen, Dung D. Le, Duy Khuong Nguyen, Viet-Anh Tran, Richard G. Baraniuk, Nhat Ho, Stanley J. Osher

Inspired by this observation, we propose Transformer with a Mixture of Gaussian Keys (Transformer-MGK), a novel transformer architecture that replaces redundant heads in transformers with a mixture of keys at each head.

Language Modelling

Fast Parallel Randomized Algorithm for Nonnegative Matrix Factorization with KL Divergence for Large Sparse Datasets

no code implementations • 14 Apr 2016 • Duy Khuong Nguyen, Tu Bao Ho

However, minimizing NMF with KL divergence is much more difficult than minimizing NMF with Frobenius norm; and sparse models, sparse representation and fast algorithms for large sparse datasets are still challenges for NMF with KL divergence.