Search Results for author:
Found 11 papers, 2 papers with code
Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval
To correlate the context of objects with the textual context, we further refine the visual semantic representation via the cross-level object-sentence and word-image based interactive attention.
Global2Local: A Joint-Hierarchical Attention for Video Captioning
Recently, automatic video captioning has attracted increasing attention, where the core challenge lies in capturing the key semantic items, like objects and actions as well as their spatial-temporal correlations from the redundant frames and semantic content.
Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation
Echocardiography is widely used to clinical practice for diagnosis and treatment, e. g., on the common congenital heart defects.
Weakly-Supervised Dense Action Anticipation
We present a (semi-) weakly supervised method using only a small number of fully-labelled sequences and predominantly sequences in which only the (one) upcoming action is labelled.
Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrieval
In this work, we propose to address the above issue from two aspects: (i) constructing intrinsic structure (along with relations) among the fragments of respective modalities, e. g., "dog $\to$ play $\to$ ball" in semantic structure for an image, and (ii) seeking explicit inter-modal structural and semantic correspondence between the visual and textual modalities.
Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network
The latter contains a Global Adaptive Controller that can adaptively fuse the global information into the decoder to guide the caption generation.
Variational Structured Semantic Inference for Diverse Image Captioning
To model these two inherent diversities in image captioning, we propose a Variational Structured Semantic Inferring model (termed VSSI-cap) executed in a novel structured encoder-inferer-decoder schema.
Semantic-aware Image Deblurring
Specially, we propose a novel Structured-Spatial Semantic Embedding model for image deblurring (termed S3E-Deblur), which introduces a novel Structured-Spatial Semantic tree model (S3-tree) to bridge two basic tasks in computer vision: image deblurring (ImD) and image captioning (ImC).
Scene-based Factored Attention for Image Captioning
Specifically, the proposed module first embeds the scene concepts into factored weights explicitly and attends the visual information extracted from the input image.
GroupCap: Group-Based Image Captioning With Structured Relevance and Diversity Constraints
In offline optimization, we adopt an end-to-end formulation, which jointly trains the visual tree parser, the structured relevance and diversity constraints, as well as the LSTM based captioning model.
