Neural Architecture Transfer

Neural architecture search (NAS) has emerged as a promising avenue for automatically designing task-specific neural networks. Most existing NAS approaches require one complete search for each deployment specification of hardware or objective... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Image Classification CIFAR-10 NAT-M3 Percentage correct 98.2 # 14
Percentage error 1.8 # 5
FLOPS 392M # 5
PARAMS 6.2M # 6
Image Classification CIFAR-10 NAT-M4 Percentage correct 98.4 # 9
Percentage error 1.6 # 4
FLOPS 468M # 6
PARAMS 6.9M # 7
Neural Architecture Search CIFAR-10 NAT-M3 Search Time (GPU days) 1.0 # 4
Parameters 6.2M # 7
FLOPS 392M # 6
Neural Architecture Search CIFAR-10 NAT-M4 Search Time (GPU days) 1.0 # 4
Parameters 6.9M # 8
FLOPS 468M # 7
Neural Architecture Search CIFAR-10 NAT-M1 Search Time (GPU days) 1.0 # 4
Parameters 4.3M # 5
FLOPS 232M # 3
Neural Architecture Search CIFAR-10 NAT-M2 Search Time (GPU days) 1.0 # 4
Parameters 4.6M # 6
FLOPS 291M # 4
Image Classification CIFAR-10 NAT-M1 Percentage correct 97.4 # 25
Percentage error 2.6 # 15
FLOPS 232M # 3
PARAMS 4.3M # 4
Image Classification CIFAR-10 NAT-M2 Percentage correct 97.9 # 18
Percentage error 2.1 # 9
FLOPS 291M # 4
PARAMS 4.6M # 5
Neural Architecture Search CIFAR-100 NAT-M1 FLOPS 261M # 2
Percentage Error 14.0 # 5
PARAMS 3.8M # 2
Neural Architecture Search CIFAR-100 NAT-M3 FLOPS 492M # 4
Percentage Error 12.3 # 2
PARAMS 7.8M # 4
Neural Architecture Search CIFAR-100 NAT-M4 FLOPS 796M # 5
Percentage Error 11.7 # 1
PARAMS 9.0M # 5
Image Classification CIFAR-100 NAT-M4 Percentage correct 88.3 # 11
Percentage error 11.7 # 6
FLOPS 796M # 6
PARAMS 9.0M # 7
Image Classification CIFAR-100 NAT-M3 Percentage correct 87.7 # 12
Percentage error 12.3 # 7
FLOPS 492M # 5
PARAMS 7.8M # 5
Image Classification CIFAR-100 NAT-M2 Percentage correct 87.5 # 13
Percentage error 12.5 # 8
FLOPS 398M # 4
PARAMS 6.4M # 4
Neural Architecture Search CIFAR-100 NAT-M2 FLOPS 398M # 3
Percentage Error 12.5 # 3
PARAMS 6.4M # 3
Image Classification CIFAR-100 NAT-M1 Percentage correct 86.0 # 16
Percentage error 14.0 # 10
FLOPS 261M # 3
PARAMS 3.8M # 3
Neural Architecture Search CIFAR-10 Image Classification NAT-M4 Percentage error 1.6 # 1
Params 6.9M # 10
FLOPS 468M # 6
Neural Architecture Search CIFAR-10 Image Classification NAT-M3 Percentage error 1.8 # 2
Params 6.2M # 9
FLOPS 392M # 5
Neural Architecture Search CIFAR-10 Image Classification NAT-M1 Percentage error 2.6 # 12
Params 4.3M # 6
FLOPS 232M # 3
Neural Architecture Search CIFAR-10 Image Classification NAT-M2 Percentage error 2.1 # 6
Params 4.6M # 7
FLOPS 291M # 4
Neural Architecture Search CINIC-10 NAT-M2 Accuracy (%) 94.1 # 3
FLOPS 411M # 2
PARAMS 6.2M # 2
Neural Architecture Search CINIC-10 NAT-M4 Accuracy (%) 94.8 # 1
FLOPS 710M # 4
PARAMS 9.1M # 4
Image Classification CINIC-10 NAT-M1 Accuracy 93.4 # 4
FLOPS 317M # 2
PARAMS 4.6M # 2
Image Classification CINIC-10 NAT-M2 Accuracy 94.1 # 3
FLOPS 411M # 3
PARAMS 6.2M # 3
Image Classification CINIC-10 NAT-M3 Accuracy 94.3 # 2
FLOPS 501M # 4
PARAMS 8.1M # 4
Image Classification CINIC-10 NAT-M4 Accuracy 94.8 # 1
FLOPS 710M # 5
PARAMS 9.1M # 5
Neural Architecture Search CINIC-10 NAT-M1 Accuracy (%) 93.4 # 4
FLOPS 317M # 1
PARAMS 4.6M # 1
Neural Architecture Search CINIC-10 NAT-M3 Accuracy (%) 94.3 # 2
FLOPS 501M # 3
PARAMS 8.1M # 3
Neural Architecture Search DTD NAT-M3 Accuracy (%) 78.4 # 2
FLOPS 347M # 3
PARAMS 4.1M # 2