Search Results for author:
Found 9 papers, 7 papers with code
Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
We combine the learning rate distributions thus found and show that they generalize to better performance with respect to the problem of catastrophic forgetting.
Debiasing Sentence Embedders through Contrastive Word Pairs
It is problematic that most debiasing approaches are directly transferred from word embeddings, therefore these approaches fail to take into account the nonlinear nature of sentence embedders and the embeddings they produce.
Neural Architecture Search for Sentence Classification with BERT
Pre training of language models on large text corpora is common practice in Natural Language Processing.
Improving Line Search Methods for Large Scale Neural Network Training
In recent studies, line search methods have shown significant improvements in the performance of traditional stochastic gradient descent techniques, eliminating the need for a specific learning rate schedule.
Faster Convergence for Transformer Fine-tuning with Line Search Methods
Recent works have shown that line search methods greatly increase performance of traditional stochastic gradient descent methods on a variety of datasets and architectures [1], [2].
Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup
We use a dataset created this way for the development and evaluation of a boolean agent RAG setup: A system in which a LLM can decide whether to query a vector database or not, thus saving tokens on questions that can be answered with internal knowledge.
Novel transfer learning schemes based on Siamese networks and synthetic data
We make the data sets available at https://pub. uni-bielefeld. de/record/2960030.
The SAME score: Improved cosine based bias score for word embeddings
Furthermore, we thoroughly investigate the existing cosine-based scores and their limitations in order to show why these scores fail to report biases in some situations.
Evaluating Metrics for Bias in Word Embeddings
However, lately some works have raised doubts about these metrics showing that even though such metrics report low biases, other tests still show biases.
