Search Results for author:
Found 18 papers, 12 papers with code
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.
Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods
In this work, we systematically study the problem of personalized text-to-image generation, where the output image is expected to portray information about specific human subjects.
Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models
Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt.
ShapeTalk: A Language Dataset and Framework for 3D Shape Edits and Deformations
In this work, we aim to facilitate the task of editing the geometry of 3D models through the use of natural language.
ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes
The two popular datasets ScanRefer [16] and ReferIt3D [3] connect natural language to real-world 3D data.
LADIS: Language Disentanglement for 3D Shape Editing
Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision.
Affection: Learning Affective Explanations for Real-World Visual Data
To embark on this journey, we introduce and share with the research community a large-scale dataset that contains emotional reactions and free-form textual explanations for 85, 007 publicly available images, analyzed by 6, 283 annotators who were asked to indicate and explain how and why they felt in a particular way when observing a specific image, producing a total of 526, 749 responses.
Quantized GAN for Complex Music Generation from Dance Videos
We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates complex musical samples conditioned on dance videos.
NeROIC: Neural Rendering of Objects from Online Image Collections
We present a novel method to acquire object representations from online image collections, capturing high-quality geometry and material properties of arbitrary objects from photographs with varying cameras, illumination, and backgrounds.
PartGlot: Learning Shape Part Segmentation from Language Reference Games
We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language.
ArtEmis: Affective Language for Visual Art
We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.
DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces
Shape deformation is an important component in any geometry processing toolbox.
Graphics
Exploring Long Tail Visual Relationship Recognition with Large Vocabulary
We use these benchmarks to study the performance of several state-of-the-art long-tail models on the LTVRR setup.
ShapeGlot: Learning Language for Shape Differentiation
We also find that these models are amenable to zero-shot transfer learning to novel object classes (e. g. transfer from training on chairs to testing on lamps), as well as to real-world images drawn from furniture catalogs.
Learning to Refer to 3D Objects with Natural Language
We further show that a neural speaker that is `listener-aware' --- that plans its utterances according to how an imagined listener would interpret its words in context --- produces more discriminative referring expressions than an `listener-unaware' speaker, as measured by human performance in identifying the correct object.
OperatorNet: Recovering 3D Shapes From Difference Operators
This paper proposes a learning-based framework for reconstructing 3D shapes from functional operators, compactly encoded as small-sized matrices.
Composite Shape Modeling via Latent Space Factorization
We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling.
Learning Representations and Generative Models for 3D Point Clouds
Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling.
