no code implementations • CLASP 2022 • Simon Dobnik, Robin Cooper, Adam Ek, Bill Noble, Staffan Larsson, Nikolai Ilinykh, Vladislav Maraev, Vidya Somashekarappa
In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the aspects of the application for which they are used.
no code implementations • PVLAM (LREC) 2022 • Nikolai Ilinykh, Rafal Černiavski, Eva Elžbieta Sventickaitė, Viktorija Buzaitė, Simon Dobnik
We conclude that face description generation systems are more susceptible to language rather than vision data augmentation.
no code implementations • INLG (ACL) 2020 • Nikolai Ilinykh, Simon Dobnik
Generating multi-sentence image descriptions is a challenging task, which requires a good model to produce coherent and accurate paragraphs, describing salient objects in the image.
Ranked #10 on Image Paragraph Captioning on Image Paragraph Captioning
1 code implementation • ACL (WebNLG, INLG) 2020 • Diego Moussallem, Paramjot Kaur, Thiago Ferreira, Chris van der Lee, Anastasia Shimorina, Felix Conrads, Michael Röder, René Speck, Claire Gardent, Simon Mille, Nikolai Ilinykh, Axel-Cyrille Ngonga Ngomo
The RDF-to-text task has recently gained substantial attention due to the continuous growth of RDF knowledge graphs in number and size.
no code implementations • ACL (WebNLG, INLG) 2020 • Thiago castro Ferreira, Claire Gardent, Nikolai Ilinykh, Chris van der Lee, Simon Mille, Diego Moussallem, Anastasia Shimorina
WebNLG+ offers two challenges: (i) mapping sets of RDF triples to English or Russian text (generation) and (ii) converting English or Russian text to sets of RDF triples (semantic parsing).
no code implementations • ACL (mmsr, IWCS) 2021 • Nikolai Ilinykh, Simon Dobnik
In this paper, we examine masked self-attention in a multi-modal transformer trained for the task of image captioning.
1 code implementation • Findings (ACL) 2022 • Nikolai Ilinykh, Simon Dobnik
We explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked self-attention (text generation) and cross-modal attention (information fusion).
1 code implementation • 7 Mar 2023 • Bill Noble, Nikolai Ilinykh
Human speakers can generate descriptions of perceptual concepts, abstracted from the instance-level.
no code implementations • 10 Sep 2021 • Simon Dobnik, Robin Cooper, Adam Ek, Bill Noble, Staffan Larsson, Nikolai Ilinykh, Vladislav Maraev, Vidya Somashekarappa
In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the aspects of the application for which they are used.
no code implementations • WS 2019 • Nikolai Ilinykh, Sina Zarrie{\ss}, David Schlangen
We present a dataset consisting of what we call image description sequences, which are multi-sentence descriptions of the contents of an image.
no code implementations • 11 Jul 2019 • Nikolai Ilinykh, Sina Zarrieß, David Schlangen
Building computer systems that can converse about their visual environment is one of the oldest concerns of research in Artificial Intelligence and Computational Linguistics (see, for example, Winograd's 1972 SHRDLU system).
no code implementations • WS 2018 • Nikolai Ilinykh, Sina Zarrie{\ss}, David Schlangen
Image captioning models are typically trained on data that is collected from people who are asked to describe an image, without being given any further task context.