Recurrent Highway Networks with Grouped Auxiliary Memory

IEEE Access 2019 Wei Luo ; Feng Yu

Recurrent neural networks (RNNs) are challenging to train, let alone those with deep spatial structures. Architectures built upon highway connections such as Recurrent Highway Network (RHN) were developed to allow larger step-to-step transition depth, leading to more expressive models... (read more)

PDF
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK BENCHMARK
Stock Trend Prediction FI-2010 BL-GAM-RHN-7 F1 (H50) 0.8088 # 1
Accuracy (H50) 0.8202 # 1
Language Modelling Penn Treebank (Character Level) GAM-RHN-5 Bit per Character (BPC) 1.147 # 1
Number of params 16.0M # 3
Sequential Image Classification Sequential MNIST GAM-RHN-1 Permuted Accuracy 96.8% # 2
Language Modelling Text8 GAM-RHN-10 Bit per Character (BPC) 1.157 # 7
Number of params 44.7M # 7

Methods used in the Paper