Hyperparameter optimization with REINFORCE and Transformers

Reinforcement Learning has yielded promising results for Neural Architecture Search (NAS). In this paper, we demonstrate how its performance can be improved by using a simplified Transformer block to model the policy network... (read more)

PDF Abstract
No code implementations yet. Submit your code now

Datasets


Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper


METHOD TYPE
Adam
Stochastic Optimization
Residual Connection
Skip Connections
Dropout
Regularization
Multi-Head Attention
Attention Modules
BPE
Subword Segmentation
Softmax
Output Functions
Dense Connections
Feedforward Networks
Label Smoothing
Regularization
Layer Normalization
Normalization
Scaled Dot-Product Attention
Attention Mechanisms
Random Search
Hyperparameter Search
Transformer
Transformers