Sign Language Translation
35 papers with code • 5 benchmarks • 13 datasets
Given a video containing sign language, the task is to predict the translation into (written) spoken language.
Image credit: How2Sign
Benchmarks
These leaderboards are used to track progress in Sign Language Translation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Datasets
WLASL
How2Sign
OpenASL
BosphorusSign22k
Content4All
WMT-SLT
ASLG-PC12
Most implemented papers
SLTUNET: A Simple Unified Model for Sign Language Translation
bzhangGo/sltunet
•
•
International Conference on Learning Representations (ICLR) 2023
We propose SLTUNET, a simple unified neural model designed to support multiple SLTrelated tasks jointly, such as sign-to-gloss, gloss-to-text and sign-to-text translation.
Cross-modality Data Augmentation for End-to-End Sign Language Translation
atrewin/signxmda
•
•
It has been a challenging task due to the modality gap between sign videos and texts and the data scarcity of labeled data.
Gloss-Free End-to-End Sign Language Translation
In this paper, we tackle the problem of sign language translation (SLT) without gloss annotations.
An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation
Sign language translation systems are complex and require many components.
ISLTranslate: Dataset for Translating Indian Sign Language
To the best of our knowledge, it is the largest translation dataset for continuous Indian Sign Language.
Gloss Attention for Gloss-free Sign Language Translation
yinaoxiong/gaslt
•
•
CVPR 2023
We find that it can provide two aspects of information for the model, 1) it can help the model implicitly learn the location of semantic boundaries in continuous sign language videos, 2) it can help the model understand the sign language video globally.
Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining
zhoubenjia/gfslt-vlp
•
•
ICCV 2023
Many previous methods employ an intermediate representation, i. e., gloss sequences, to facilitate SLT, thus transforming it into a two-stage task of sign language recognition (SLR) followed by sign language translation (SLT).
SignBank+: Preparing a Multilingual Sign Language Dataset for Machine Translation Using Large Language Models
We introduce SignBank+, a clean version of the SignBank dataset, optimized for machine translation between spoken language text and SignWriting, a phonetic sign language writing system.
JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing
Advancements in sign language processing have been hindered by a lack of sufficient data, impeding progress in recognition, translation, and production tasks.
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models
pooyafayyaz/fingerspelling-posenet
•
•
We also propose a novel two-stage inference approach that re-ranks the hypotheses using the language model capabilities of the decoder.
