Search Results for author:
Found 8 papers, 1 papers with code
Lessons Learned from GPT-SW3: Building the First Large-Scale Generative Language Model for Swedish
We present GTP-SW3, a 3. 5 billion parameter autoregressive language model, trained on a newly created 100 GB Swedish corpus.
Continual Learning Under Language Shift
The recent increase in data and model scale for language model pre-training has led to huge training costs.
GPT-SW3: An Autoregressive Language Model for the Nordic Languages
This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3.
The Nordic Pile: A 1.2TB Nordic Dataset for Language Modeling
Pre-training Large Language Models (LLMs) require massive amounts of text data, and the performance of the LLMs typically correlates with the scale and quality of the datasets.
Cross-lingual Transfer of Monolingual Models
Additionally, the results of evaluating the transferred models in source language tasks reveal that their performance in the source domain deteriorates after transfer.
Predicting Treatment Outcome from Patient Texts:The Case of Internet-Based Cognitive Behavioural Therapy
We investigate the feasibility of applying standard text categorisation methods to patient text in order to predict treatment outcome in Internet-based cognitive behavioural therapy.
Deep Representational Re-tuning using Contrastive Tension
Extracting semantically useful natural language sentence representations from pre-trained deep neural networks such as Transformers remains a challenge.
SenseCluster at SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
We (Team Skurt) propose a simple method to detect lexical semantic change by clustering contextualized embeddings produced by XLM-R, using K-Means++.
