Search Results for author:
Found 19 papers, 13 papers with code
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility.
Variational Learning is Effective for Large Deep Networks
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks.
SegRCDB: Semantic Segmentation via Formula-Driven Supervised Learning
SegRCDB has a high potential to contribute to semantic segmentation pre-training and investigation by enabling the creation of large datasets without manual annotation.
Pre-training Vision Transformers with Very Limited Synthesized Images
Prior work on FDSL has shown that pre-training vision transformers on such synthetic datasets can yield competitive accuracy on a wide range of downstream tasks.
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Gradient preconditioning is a key technique to integrate the second-order information into gradients for improving and extending gradient-based learning algorithms.
Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves
Unlike JFT-300M which is a static dataset, the quality of synthetic datasets will continue to improve, and the current work is a testament to this possibility.
Informative Sample-Aware Proxy for Deep Metric Learning
Among various supervised deep metric learning methods proxy-based approaches have achieved high retrieval accuracies.
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Modern deep learning systems do not generalize well when the test data distribution is slightly different to the training data distribution.
Replacing Labeled Real-image Datasets with Auto-generated Contours
In the present work, we show that the performance of formula-driven supervised learning (FDSL) can match or even exceed that of ImageNet-21k without the use of real images, human-, and self-supervision during the pre-training of Vision Transformers (ViTs).
Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
Generalization measures are intensively studied in the machine learning community for better modeling generalization gaps.
OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
Inverse Reinforcement Learning (IRL) is attractive in scenarios where reward engineering can be tedious.
RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering
Furthermore, we utilize differentiable Levenberg-Marquardt (LM) optimization to refine a pose fast and accurately by minimizing the feature-metric error between the input and rendered image representations without the need of zooming in.
Ranked #5 on
6D Pose Estimation using RGB
on LineMOD
Epipolar-Guided Deep Object Matching for Scene Change Detection
To cope with the difficulty, we introduce a deep graph matching network that establishes object correspondence between an image pair.
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Large-scale distributed training of deep neural networks results in models with worse generalization performance as a result of the increase in the effective mini-batch size.
Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis
Hierarchical Matrix (H-matrix) is an approximation technique which splits a target dense matrix into multiple submatrices, and where a selected portion of submatrices are low-rank approximated.
Mathematical Software Distributed, Parallel, and Cluster Computing
Practical Deep Learning with Bayesian Principles
Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted.
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size.
Extreme Scale FMM-Accelerated Boundary Integral Equation Solver for Wave Scattering
With distributed memory optimizations, on the other hand, we report near-optimal efficiency in the weak scalability study with respect to both the logarithmic communication complexity as well as the theoretical scaling complexity of FMM.
Performance Computational Engineering, Finance, and Science Mathematical Software
Asynchronous Execution of the Fast Multipole Method Using Charm++
Fast multipole methods (FMM) on distributed mem- ory have traditionally used a bulk-synchronous model of com- municating the local essential tree (LET) and overlapping it with computation of the local data.
Distributed, Parallel, and Cluster Computing 70F10 D.1.2; D.1.3; G.1.0; G.1.2
