Search Results for author:
Found 43 papers, 16 papers with code
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback
Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation's aesthetic quality.
Substance or Style: What Does Your Image Embedding Know?
We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).
LayerNAS: Neural Architecture Search in Polynomial Complexity
For a model architecture with $L$ layers, we perform layerwise-search for each layer, selecting from a set of search options $\mathbb{S}$.
RARR: Researching and Revising What Language Models Say, Using Language Models
Language models (LMs) now excel at many tasks such as few-shot learning, question answering, reasoning, and dialog.
Representative Image Feature Extraction via Contrastive Learning Pretraining for Chest X-ray Report Generation
Medical report generation is a challenging task since it is time-consuming and requires expertise from experienced radiologists.
Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios.
Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search
Emerging device-based Computing-in-memory (CiM) has been proved to be a promising candidate for high-energy efficiency deep neural network (DNN) computations.
CARLS: Cross-platform Asynchronous Representation Learning System
In this work, we propose CARLS, a novel framework for augmenting the capacity of existing deep learning frameworks by enabling multiple components -- model trainers, knowledge makers and knowledge banks -- to concertedly work together in an asynchronous fashion across hardware platforms.
OmniNet: Omnidirectional Representations from Transformers
In OmniNet, instead of maintaining a strictly horizontal receptive field, each token is allowed to attend to all tokens in the entire network.
Ranked #1 on
Machine Translation
on WMT2017 Russian-English
Switch Spaces: Learning Product Spaces with Sparse Gating
In order for learned representations to be effective and efficient, it is ideal that the geometric inductive bias aligns well with the underlying structure of the data.
Synthesizer: Rethinking Self-Attention for Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
Context-Aware Temperature for Language Modeling
Current practices to apply temperature scaling assume either a fixed, or a manually-crafted dynamically changing schedule.
HyperGrid Transformers: Towards A Single Model for Multiple Tasks
Specifically, we propose a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Graph Autoencoders with Deconvolutional Networks
Recent studies have indicated that Graph Convolutional Networks (GCNs) act as a \emph{low pass} filter in spectral domain and encode smoothed node representations.
Mitigating Forgetting in Online Continual Learning via Instance-Aware Parameterization
To preserve the knowledge we learn from previous instances, we proposed a method to protect the path by restricting the gradient updates of one instance from overriding past updates calculated from previous instances if these instances are not similar.
Adversarial Robustness Across Representation Spaces
In this work we extend the above setting to consider the problem of training of deep neural networks that can be made simultaneously robust to perturbations applied in multiple natural representation spaces.
AirConcierge: Generating Task-Oriented Dialogue via Efficient Large-Scale Knowledge Retrieval
Despite recent success in neural task-oriented dialogue systems, developing such a real-world system involves accessing large-scale knowledge bases (KBs), which cannot be simply encoded by neural approaches, such as memory network mechanisms.
Question Answering with Long Multiple-Span Answers
To this end, we present MASH-QA, a Multiple Answer Spans Healthcare Question Answering dataset from the consumer health domain, where answers may need to be excerpted from multiple, non-consecutive parts of text spanned across a long document.
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
The proposed approach is based on a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Remix: Rebalanced Mixup
In this work, we propose a new regularization technique, Remix, that relaxes Mixup's formulation and enables the mixing factors of features and labels to be disentangled.
Robust Processing-In-Memory Neural Networks via Noise-Aware Normalization
In this paper, we propose a noise-agnostic method to achieve robust neural network performance against any noise setting.
Synthesizer: Rethinking Self-Attention in Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
Ranked #1 on
Dialogue Generation
on Persona-Chat
(BLEU-1 metric, using extra
training data)
Low-Dimensional Hyperbolic Knowledge Graph Embeddings
However, existing hyperbolic embedding methods do not account for the rich logical patterns in KGs.
Ranked #5 on
Link Prediction
on YAGO3-10
Sparse Sinkhorn Attention
We propose Sparse Sinkhorn Attention, a new efficient and sparse method for learning to attend.
Learning with Hierarchical Complement Objective
Label hierarchies widely exist in many vision-related problems, ranging from explicit label hierarchies existed in image classification to latent label hierarchies existed in semantic segmentation.
Natural Adversarial Sentence Generation with Gradient-based Perturbation
This work proposes a novel algorithm to generate natural language adversarial input for text classification models, in order to investigate the robustness of these models.
On the Robustness of Self-Attentive Models
This work examines the robustness of self-attentive neural networks against adversarial input perturbations.
A2N: Attending to Neighbors for Knowledge Graph Inference
State-of-the-art models for knowledge graph completion aim at learning a fixed embedding representation of entities in a multi-relational graph which can generalize to infer unseen entity relationships at test time.
COCO-GAN: Conditional Coordinate Generative Adversarial Network
The fact that the patch generation process is independent to each other inspires a wide range of new applications: firstly, "Patch-Inspired Image Generation" enables us to generate the entire image based on a single patch.
COCO-GAN: Generation by Parts via Conditional Coordinating
On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides high-parallelism, and can generate parts of images on-demand.
Ranked #1 on
Image Generation
on CelebA-HQ 64x64
Improving Adversarial Robustness via Guided Complement Entropy
Adversarial robustness has emerged as an important topic in deep learning as carefully crafted attack samples can significantly disturb the performance of a model.
Complement Objective Training
Although being a widely-adopted approach, using cross entropy as the primary objective exploits mostly the information from the ground-truth class for maximizing data likelihood, and largely ignores information from the complement (incorrect) classes.
Graph-RISE: Graph-Regularized Image Semantic Embedding
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering.
Ranked #11 on
Image Classification
on iNaturalist
InstaNAS: Instance-aware Neural Architecture Search
Conventional Neural Architecture Search (NAS) aims at finding a single architecture that achieves the best performance, which usually optimizes task related learning objectives such as accuracy.
Searching Toward Pareto-Optimal Device-Aware Neural Architectures
Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performance in many tasks such as image classification and language understanding.
Escaping from Collapsing Modes in a Constrained Space
Generative adversarial networks (GANs) often suffer from unpredictable mode-collapsing during training.
Ranked #26 on
Image Generation
on CelebA 64x64
MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures.
DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
We propose DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures, optimizing for both device-related (e. g., inference time and memory usage) and device-agnostic (e. g., accuracy and model size) objectives.
HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks
While selecting the hyper-parameters of Neural Networks (NNs) has been so far treated as an art, the emergence of more complex, deeper architectures poses increasingly more challenges to designers and Machine Learning (ML) practitioners, especially when power and memory constraints need to be considered.
NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks
We also propose the "energy-precision ratio" (EPR) metric to guide machine learners in selecting an energy-efficient CNN architecture that better trades off the energy consumption and prediction accuracy.
DC-Prophet: Predicting Catastrophic Machine Failures in DataCenters
When will a server fail catastrophically in an industrial datacenter?
