Search Results for author:
Found 34 papers, 15 papers with code
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
The mixture proportions of pretraining data domains (e. g., Wikipedia, books, web text) greatly affect language model (LM) performance.
Multimodal contrastive learning for diagnosing cardiovascular diseases from electrocardiography (ECG) signals and patient metadata
This work discusses the use of contrastive learning and deep learning for diagnosing cardiovascular diseases from electrocardiography (ECG) signals.
Ensemble Learning of Myocardial Displacements for Myocardial Infarction Detection in Echocardiography
The proposed approach demonstrated excellent performance in detecting MI.
Symbolic Discovery of Optimization Algorithms
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
MixupE: Understanding and Improving Mixup from Directional Derivative Perspective
Mixup is a popular data augmentation technique for training deep neural networks where additional samples are generated by linearly interpolating pairs of inputs and their labels.
Combined Scaling for Zero-shot Transfer Learning
Second, while increasing the dataset size and the model size has been the defacto method to improve the performance of deep learning models like BASIC, the effect of a large contrastive batch size on such contrastive-trained image-text models is not well-understood.
An Applied Deep Learning Approach for Estimating Soybean Relative Maturity from UAV Imagery to Aid Plant Breeding Decisions
Specifically for soybeans, identifying their relative maturity is a vital piece of information used for advancement decisions.
WheatNet: A Lightweight Convolutional Neural Network for High-throughput Image-based Wheat Head Detection and Counting
To help mitigate this data collection bottleneck in wheat breeding, we propose a novel deep learning framework to accurately and efficiently count wheat heads to aid in the gathering of real-time data for decision making.
Meta Back-translation
Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data.
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
In this paper, we leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps in the Conceptual Captions dataset.
Ranked #1 on
Image Classification
on VTAB-1k
(using extra training data)
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks
As a result, these conventional methods are less effective than methods that leverage the structures, such as SpatialDropout and DropBlock, which randomly drop the values at certain contiguous areas in the hidden states and setting them to zero.
Ranked #1 on
Image Classification
on cifar-10,4000
Simultaneous Corn and Soybean Yield Prediction from Remote Sensing Data Using Deep Transfer Learning
A model that predicts the yield of multiple crops and concurrently considers the interaction between multiple crop yields.
Towards Domain-Agnostic Contrastive Learning
Despite recent success, most contrastive self-supervised learning methods are domain-specific, relying heavily on data augmentation techniques that require knowledge about a particular domain, such as image cropping and rotation.
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks.
High-Throughput Image-Based Plant Stand Count Estimation Using Convolutional Neural Networks
The future landscape of modern farming and plant breeding is rapidly changing due to the complex needs of our society.
Image-Based Sorghum Head Counting When You Only Look Once
Modern trends in digital agriculture have seen a shift towards artificial intelligence for crop quality assessment and yield estimation.
DeepCorn: A Semi-Supervised Deep Learning Method for High-Throughput Image-Based Corn Kernel Counting and Yield Estimation
In this paper, we propose a novel deep learning method for counting on-ear corn kernels in-field to aid in the gathering of real-time data and, ultimately, to improve decision making to maximize yield.
Convolutional Neural Networks for Image-based Corn Kernel Detection and Counting
The sliding window approach uses a convolutional neural network (CNN) for kernel detection.
Meta Pseudo Labels
We present Meta Pseudo Labels, a semi-supervised learning method that achieves a new state-of-the-art top-1 accuracy of 90. 2% on ImageNet, which is 1. 6% better than the existing state-of-the-art.
Optimizing Data Usage via Differentiable Rewards
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems.
Semi-supervised Learning by Coaching
Recent semi-supervised learning (SSL) methods often have a teacher to train a student in order to propagate labels from labeled data to unlabeled data.
Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems
To this end, an optimization-based nested algorithm that considers tuning hyperparameters as well as finding the optimal weights to combine ensembles (Generalized Weighted Ensemble with Internally Tuned Hyperparameters (GEM-ITH)) is designed.
Multilingual Neural Machine Translation With Soft Decoupled Encoding
Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages.
A Tree-based Decoder for Neural Machine Translation
Recent advances in Neural Machine Translation (NMT) show that adding syntactic information to NMT systems can improve the quality of their translations.
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation
In this work, we examine methods for data augmentation for text-based tasks such as neural machine translation (NMT).
Efficient Neural Architecture Search via Parameters Sharing
We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design.
Ranked #32 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Efficient Neural Architecture Search via Parameter Sharing
The controller is trained with policy gradient to select a subgraph that maximizes the expected reward on the validation set.
Faster Discovery of Neural Architectures by Searching for Paths in a Large Model
We propose Efficient Neural Architecture Search (ENAS), a faster and less expensive approach to automated model design than previous methods.
A Hierarchical Model for Device Placement
We introduce a hierarchical model for efficient placement of computational graphs onto hardware devices, especially in heterogeneous environments with a mixture of CPUs, GPUs, and other computational devices.
Device Placement Optimization with Reinforcement Learning
Key to our method is the use of a sequence-to-sequence model to predict which subsets of operations in a TensorFlow graph should run on which of the available devices.
Neural Combinatorial Optimization with Reinforcement Learning
Despite the computational expense, without much engineering and heuristic designing, Neural Combinatorial Optimization achieves close to optimal results on 2D Euclidean graphs with up to 100 nodes.
Effective Approaches to Attention-based Neural Machine Translation
Our ensemble model using different attention architectures has established a new state-of-the-art result in the WMT'15 English to German translation task with 25. 9 BLEU points, an improvement of 1. 0 BLEU points over the existing best system backed by NMT and an n-gram reranker.
Ranked #1 on
Machine Translation
on 20NEWS
(Accuracy metric)
