Search Results for author:
Found 14 papers, 6 papers with code
Animate Your Motion: Turning Still Images into Dynamic Videos
For the first time, we integrate both semantic and motion cues within a diffusion model for video generation, as demonstrated in Fig 1.
NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties
In the pursuit to understand the intricacies of human brain's visual processing, reconstructing dynamic visual experiences from brain activities emerges as a challenging yet fascinating endeavor.
Generating Explanations in Medical Question-Answering by Expectation Maximization Inference over Evidence
Medical Question Answering~(medical QA) systems play an essential role in assisting healthcare workers in finding answers to their questions.
Decoding Realistic Images from Brain Activity with Contrastive Self-supervision and Latent Diffusion
Reconstructing visual stimuli from human brain activities provides a promising opportunity to advance our understanding of the brain's visual system and its connection with computer vision models.
Elucidating the Exposure Bias in Diffusion Models
In this paper, we systematically investigate the exposure bias problem in diffusion models by first analytically modelling the sampling distribution, based on which we then attribute the prediction error at each sampling step as the root cause of the exposure bias issue.
Ranked #9 on
Image Generation
on CIFAR-10
Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities
The second phase tunes the feature learner to attend to neural activation patterns most informative for visual reconstruction with guidance from an image auto-encoder.
Ranked #1 on
Brain Visual Reconstruction from fMRI
on GOD
Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps
In this work, we conduct a systematic study of exposure bias in DPM and, intriguingly, we find that the exposure bias could be alleviated with a novel sampling method that we propose, without retraining the model.
Crossword: A Semantic Approach to Data Compression via Masking
The traditional methods for data compression are typically based on the symbol-level statistics, with the information source modeled as a long sequence of i. i. d.
Layout-aware Dreamer for Embodied Referring Expression Grounding
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction.
Find a Way Forward: a Language-Guided Semantic Map Navigator
In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map.
Modeling Coreference Relations in Visual Dialog
Visual dialog is a vision-language task where an agent needs to answer a series of questions grounded in an image based on the understanding of the dialog history and the image.
Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (VQA) is a vision-language task that requires an agent to correctly answer image-related questions using knowledge that is not presented in the given image.
Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win
The lottery ticket hypothesis states that sparse subnetworks exist in randomly initialized dense networks that can be trained to the same accuracy as the dense network they reside in.
Multiscale Dynamic Human Mobility Flow Dataset in the U.S. during the COVID-19 Epidemic
Understanding dynamic human mobility changes and spatial interaction patterns at different geographic scales is crucial for monitoring and measuring the impacts of non-pharmaceutical interventions (such as stay-at-home orders) during the pandemic.
Social and Information Networks Physics and Society
