Search Results for author:
Found 45 papers, 20 papers with code
CoCGAN: Contrastive Learning for Adversarial Category Text Generation
The discriminator of CoCGAN discriminates the authenticity of given samples and optimizes a contrastive learning objective to capture both more flexible data-to-class relations and data-to-data relations among training samples.
Semantic-Preserving Abstractive Text Summarization with Siamese Generative Adversarial Net
We propose a novel siamese generative adversarial net for abstractive text summarization (SSPGAN), which can preserve the main semantics of the source text.
Hierarchical Multi-label Text Classification with Horizontal and Vertical Category Correlations
Hierarchical multi-label text classification (HMTC) deals with the challenging task where an instance can be assigned to multiple hierarchically structured categories at the same time.
Multi Label Text Classification
Multi-Label Text Classification
+1
On decoder-only architecture for speech-to-text and large language model integration
Large language models (LLMs) have achieved remarkable success in the field of natural language processing, enabling better human-computer interaction using natural language.
Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution
We evaluate the existing end-to-end methods for VIE on the proposed dataset and observe that the performance of these methods has a distinguishable drop from SROIE (a widely used English dataset) to our proposed dataset due to the larger variance of layout and entities.
Multi-Space Neural Radiance Fields
Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects, often resulting in blurry or distorted rendering.
Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections
To remedy this issue, we present a novel surface reconstruction framework, NeuS-HSR, based on implicit neural rendering.
Collaborative Noisy Label Cleaner: Learning Scene-aware Trailers for Multi-modal Highlight Detection in Movies
Based on existing efforts, this work has two observations: (1) For different annotators, labeling highlight has uncertainty, which leads to inaccurate and time-consuming annotations.
Turning a CLIP Model into a Scene Text Detector
Recently, pretraining approaches based on vision language models have made effective progresses in the field of text detection.
OSAN: A One-Stage Alignment Network To Unify Multimodal Alignment and Unsupervised Domain Adaptation
An intuitive way to handle these two problems is to fulfill these tasks in two separate stages: aligning modalities followed by domain adaptation, or vice versa.
Consistent Depth Prediction for Transparent Object Reconstruction from RGB-D Camera
We compare the metrics on our dataset and SLAM reconstruction results in both synthetic scenes and real scenes with the previous methods.
NewsNet: A Novel Dataset for Hierarchical Temporal Segmentation
Temporal video segmentation is the get-to-go automatic video analysis, which decomposes a long-form video into smaller components for the following-up understanding tasks.
FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning
Most existing methods focus on learning noise-robust models from web images while neglecting the performance drop caused by the differences between web domain and real-world domain.
SLAN: Self-Locator Aided Network for Cross-Modal Understanding
By aggregating cross-modal information, the region filter selects key regions and the region adaptor updates their coordinates with text guidance.
Grafting Pre-trained Models for Multimodal Headline Generation
In this paper, we propose a novel approach to graft the video encoder from the pre-trained video-language model on the generative pre-trained language model.
Leveraging Key Information Modeling to Improve Less-Data Constrained News Headline Generation via Duality Fine-Tuning
Recent language generative models are mostly trained on large-scale datasets, while in some real scenarios, the training datasets are often expensive to obtain and would be small-scale.
TaCo: Textual Attribute Recognition via Contrastive Learning
To realize this goal, we design the learning paradigm from three perspectives: 1) generating attribute views, 2) extracting subtle but crucial details, and 3) exploiting valued view pairs for learning, to fully unlock the pre-training potential.
VLMAE: Vision-Language Masked Autoencoder
Image and language modeling is of crucial importance for vision-language pre-training (VLP), which aims to learn multi-modal representations from large-scale paired image-text data.
See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval
To explore the fine-grained alignment, we further propose two implicit semantic alignment paradigms: multi-level alignment (MLA) and bidirectional mask modeling (BMM).
GMN: Generative Multi-modal Network for Practical Document Information Extraction
Document Information Extraction (DIE) has attracted increasing attention due to its various advanced applications in the real world.
Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer
Specifically, our method exploits multi-modal knowledge of image-text pairs based on a vision and language pre-training (VLP) model.
Ranked #1 on
Multi-label zero-shot learning
on Open Images V4
OS-MSL: One Stage Multimodal Sequential Link Framework for Scene Segmentation and Classification
In this paper, from an alternate perspective to overcome the above challenges, we unite these two tasks into one task by a new form of predicting shots link: a link connects two adjacent shots, indicating that they belong to the same scene or category.
Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization
Open-vocabulary object detection (OVD) aims to scale up vocabulary size to detect objects of novel categories beyond the training vocabulary.
Ranked #12 on
Open Vocabulary Object Detection
on LVIS v1.0
RAAT: Relation-Augmented Attention Transformer for Relation Modeling in Document-Level Event Extraction
To further leverage relation information, we introduce a separate event relation prediction task and adopt multi-task learning method to explicitly enhance event extraction performance.
Ranked #1 on
Document-level Event Extraction
on ChFinAnn
Contrastive Graph Multimodal Model for Text Classification in Videos
However, to our knowledge, there is no existing work focused on the second step of video text classification, which will limit the guidance to downstream tasks such as video indexing and browsing.
Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation
The task of Grammatical Error Correction (GEC) has received remarkable attention with wide applications in Natural Language Processing (NLP) in recent years.
Scene Consistency Representation Learning for Video Scene Segmentation
Additionally, we suggest a more fair and reasonable benchmark to evaluate the performance of Video Scene Segmentation methods.
Relational Representation Learning in Visually-Rich Documents
To deal with the unpredictable definition of relations, we propose a novel contrastive learning task named Relational Consistency Modeling (RCM), which harnesses the fact that existing relations should be consistent in differently augmented positive views.
The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training
The self-supervised Masked Image Modeling (MIM) schema, following "mask-and-reconstruct" pipeline of recovering contents from masked image, has recently captured the increasing interest in the multimedia community, owing to the excellent ability of learning visual representation from unlabeled data.
Knowledge Mining with Scene Text for Fine-Grained Recognition
Recently, the semantics of scene text has been proven to be essential in fine-grained image classification.
Interactive Style Transfer: All is Your Palette
Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image.
Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognition
And then a spatio-temporal tuples self-attention module is proposed to capture the relationship of different joints in consecutive frames.
HybridCR: Weakly-Supervised 3D Point Cloud Semantic Segmentation via Hybrid Contrastive Regularization
To address the huge labeling cost in large-scale point cloud semantic segmentation, we propose a novel hybrid contrastive regularization (HybridCR) framework in weakly-supervised setting, which obtains competitive performance compared to its fully-supervised counterpart.
Head and Body: Unified Detector and Graph Network for Person Search in Media
It is by far the largest dataset for person search in media.
Neural Collaborative Graph Machines for Table Structure Recognition
We also show that the proposed NCGM can modulate collaborative pattern of different modalities conditioned on the context of intra-modality cues, which is vital for diversified table cases.
Ranked #6 on
Table Recognition
on PubTabNet
NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition
For the sake of trade-off between efficiency and performance, a group of works merely perform SA operation within local patches, whereas the global contextual information is abandoned, which would be indispensable for visual recognition tasks.
FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection
We convert a lightweight FCOSR model to TensorRT format, which achieves 73. 93 mAP on DOTA1. 0 at a speed of 10. 68 FPS on Jetson Xavier NX with single scale.
Interpreting BERT architecture predictions for peptide presentation by MHC class I proteins
In particular, we find that amino acids close to the peptides' N- and C-terminals are highly relevant.
Transfusion: A Novel SLAM Method Focused on Transparent Objects
In this paper, we propose a novel SLAM approach called transfusion that allows transparent object existence and recovery in the video input.
EDN: Salient Object Detection via Extremely-Downsampled Network
In this paper, we tap into this gap and show that enhancing high- level features is essential for SOD as well.
PuzzleNet: Scene Text Detection by Segment Context Graph Learning
Recently, a series of decomposition-based scene text detection methods has achieved impressive progress by decomposing challenging text regions into pieces and linking them in a bottom-up manner.
Scoot: A Perceptual Metric for Facial Sketches
In this paper, we design a perceptual metric, called Structure Co-Occurrence Texture (Scoot), which simultaneously considers the block-level spatial structure and co-occurrence texture statistics.
Sequence-based Person Attribute Recognition with Joint CTC-Attention Model
Attribute recognition has become crucial because of its wide applications in many computer vision tasks, such as person re-identification.
Enhanced-alignment Measure for Binary Foreground Map Evaluation
The existing binary foreground map (FM) measures to address various types of errors in either pixel-wise or structural ways.
Face Sketch Synthesis Style Similarity:A New Structure Co-occurrence Texture Measure
However, human perception of the similarity of two sketches will consider both structure and texture as essential factors and is not sensitive to slight ("pixel-level") mismatches.
