Search Results for author:
Found 13 papers, 9 papers with code
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging
The transformer architecture by Vaswani et al. (2017) is now ubiquitous across application domains, from natural language processing to speech processing and image understanding.
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Large language models (LLMs) can potentially democratize access to medical knowledge.
Ranked #1 on
Multiple Choice Question Answering (MCQA)
on MedMCQA
(Dev Set (Acc-%) metric)
Conditional Text Generation
Multiple Choice Question Answering (MCQA)
DoGE: Domain Reweighting with Generalization Estimation
Moreover, aiming to generalize to out-of-domain target tasks, which is unseen in the pretraining corpus (OOD domain), DoGE can effectively identify inter-domain dependencies, and consistently achieves better test perplexity on the target domain.
CoTFormer: More Tokens With Attention Make Up For Less Depth
The race to continually develop ever larger and deeper foundational models is underway.
Faster Causal Attention Over Large Sequences Through Sparse Flash Attention
While many works have proposed schemes to sparsify the attention patterns and reduce the computational overhead of self-attention, those are often limited by implementations concerns and end up imposing a simple and static structure over the attention matrix.
A Primal-dual Approach for Solving Variational Inequalities with General-form Constraints
We prove its convergence and show that the gap function of the last iterate of this inexact-ACVI method decreases at a rate of $\mathcal{O}(\frac{1}{\sqrt{K}})$ when the operator is $L$-Lipschitz and monotone, provided that the errors decrease at appropriate rates.
Improving Generalization via Uncertainty Driven Perturbations
We show that UDP is guaranteed to achieve the maximum margin decision boundary on linear models and that it notably increases it on challenging simulated datasets.
Agree to Disagree: Diversity through Disagreement for Better Transferability
This behavior can hinder the transferability of trained models by (i) favoring the learning of simpler but spurious features -- present in the training data but absent from the test data -- and (ii) by only leveraging a small subset of predictive features.
The Peril of Popular Deep Learning Uncertainty Estimation Methods
Secondly, we show on a 2D toy example that both BNNs and MCDropout do not give high uncertainty estimates on OOD samples.
Improved Generalization-Robustness Trade-off via Uncertainty Targeted Attacks
The deep learning models' sensitivity to small input perturbations raises security concerns and limits their use for applications where reliability is critical.
Taming GANs with Lookahead-Minmax
Generative Adversarial Networks are notoriously challenging to train.
Better Word Embeddings by Disentangling Contextual n-Gram Information
Pre-trained word vectors are ubiquitous in Natural Language Processing applications.
Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
The recent tremendous success of unsupervised word embeddings in a multitude of applications raises the obvious question if similar methods could be derived to improve embeddings (i. e. semantic representations) of word sequences as well.
