Search Results for author:
Found 40 papers, 11 papers with code
TextSquare: Scaling up Text-Centric Visual Instruction Tuning
Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data.
Progressive Multi-modal Conditional Prompt Tuning
Initialization is responsible for encoding image and text using a VLM, followed by a feature filter that selects text features similar to image.
Integration of Self-Supervised BYOL in Semi-Supervised Medical Image Recognition
Image recognition techniques heavily rely on abundant labeled data, particularly in medical contexts.
TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding
The image overview stage provides a comprehensive understanding of the global scene information, and the coarse localization stage approximates the image area containing the answer based on the question asked.
Enhanced Radar Perception via Multi-Task Learning: Towards Refined Data for Sensor Fusion Applications
Radar and camera fusion yields robustness in perception tasks by leveraging the strength of both sensors.
DeepEraser: Deep Iterative Context Mining for Generic Text Eraser
In this work, we present DeepEraser, an effective deep network for generic text removal.
UCE-FID: Using Large Unlabeled, Medium Crowdsourced-Labeled, and Small Expert-Labeled Tweets for Foodborne Illness Detection
To overcome these challenges, we propose EGAL, a deep learning framework for foodborne illness detection that uses small expert-labeled tweets augmented by crowdsourced-labeled and massive unlabeled data.
Scalable AI Generative Content for Vehicular Network Semantic Communication
Experimental results suggest that the proposed method surpasses the baseline in perceiving vehicles in blind spots and effectively compresses communication data.
Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs
Moreover, we curate a collection of text-rich images and prompt the text-only GPT-4 to generate 12K high-quality conversations, featuring textual locations within text-rich scenarios.
DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding
This work presents DocPedia, a novel large multimodal model (LMM) for versatile OCR-free document understanding, capable of parsing images up to 2, 560$\times$2, 560 resolution.
Progressive Recurrent Network for Shadow Removal
To handle this issue, we consider removing the shadow in a coarse-to-fine fashion and propose a simple but effective Progressive Recurrent Network (PRNet).
Sign Language Translation with Iterative Prototype
Technically, IP-SLT consists of feature extraction, prototype initialization, and iterative prototype refinement.
Ranked #5 on
Sign Language Translation
on CSL-Daily
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding
However, existing advanced algorithms are limited to effectively utilizing the immense representation capabilities and rich world knowledge inherent to these large pre-trained models, and the beneficial connections among tasks within the context of text-rich scenarios have not been sufficiently explored.
SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning
To make the best of such rectification cues, we introduce SimFIR, a simple framework for fisheye image rectification based on self-supervised representation learning.
Mobile Supply: The Last Piece of Jigsaw of Recommender System
In order to address the problem of pagination trigger mechanism, we propose a completely new module in the pipeline of recommender system named Mobile Supply.
ESMC: Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint
We propose a novel Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint (ESMC) and two alternatives: Entire Space Multi-Task Model with Siamese Network (ESMS) and Entire Space Multi-Task Model in Global Domain (ESMG) to address the PSC issue.
Multi-Task Cross-Modality Attention-Fusion for 2D Object Detection
In addition, we introduce a Multi-Task Cross-Modality Attention-Fusion Network (MCAF-Net) for object detection, which includes two new fusion blocks.
Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions
We provide a gradient backpropagation highway for low-rank adapters which eliminates the need for expensive backpropagation through the frozen pre-trained model, resulting in substantial savings of training memory and training time.
Active RIS-Assisted mmWave Indoor Signal Enhancement Based on Transparent RIS
In this paper, a novel RIS-assisted mmWave indoor enhancement scheme is proposed, in which a transparent RIS is deployed on the glass to enhance mmWave indoor signals, and three assisted transmission scenarios, namely passive RIS (PRIS), active RIS (ARIS), and a novel hybrid RIS (HRIS) are proposed.
Model-Based Monitoring and State Estimation for Digital Twins: The Kalman Filter
A digital twin (DT) monitors states of the physical twin (PT) counterpart and provides a number of benefits such as advanced visualizations, fault detection capabilities, and reduced maintenance cost.
mmWave RIS Phase Shift Feedback Based on Knowledge Base Autoencoder Framework
Then the knowledge base vectors index is obtained by calculating the similarity between feature vectors and knowledge base vectors and transmitted to the RIS.
Diffusion Probabilistic Model Based Accurate and High-Degree-of-Freedom Metasurface Inverse Design
Inverse design methods based on optimization algorithms, such as evolutionary algorithms, and topological optimizations, have been introduced to design metamaterials.
DocMAE: Document Image Rectification via Self-supervised Representation Learning
Tremendous efforts have been made on document image rectification, but how to learn effective representation of such distorted images is still under-explored.
Deep Unrestricted Document Image Rectification
To our best knowledge, this is the first learning-based method for the rectification of unrestricted document images.
Ranked #1 on
Local Distortion
on DocUNet
PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation
Our proposed framework provides a conditional feature extraction module first to extract the coarse yet effective spatiotemporal dependencies from conditional information as the global context prior.
Recurrent Generic Contour-based Instance Segmentation with Progressive Learning
It maintains a single estimate of the contour that is progressively deformed toward the object boundary.
Ranked #1 on
Semantic Contour Prediction
on Sbd val
Geometric Representation Learning for Document Image Rectification
In document image rectification, there exist rich geometric constraints between the distorted image and the ground truth one.
Utilizing Explainable AI for improving the Performance of Neural Networks
To address this, Explainable Artificial Intelligence (XAI) has been developing as a field that aims to improve the transparency of the model and increase their trustworthiness.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
Towards Efficient Modularity in Industrial Drying: A Combinatorial Optimization Viewpoint
The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies.
Knowledge Distillation based Contextual Relevance Matching for E-commerce Product Search
In this paper, we design an efficient knowledge distillation framework for e-commerce relevance matching to integrate the respective advantages of Transformer-style models and classical relevance matching models.
TWEET-FID: An Annotated Dataset for Multiple Foodborne Illness Detection Tasks
To accelerate the development of machine learning-based models for foodborne outbreak detection, we thus present TWEET-FID (TWEET-Foodborne Illness Detection), the first publicly available annotated dataset for multiple foodborne illness incident detection tasks.
Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition
Gesture recognition is one of the most intuitive ways of interaction and has gathered particular attention for human computer interaction.
DocScanner: Robust Document Image Rectification with Progressive Learning
The iterative refinements make DocScanner converge to a robust and superior rectification performance, while the lightweight recurrent architecture ensures the running efficiency.
DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction
Specifically, DocTr consists of a geometric unwarping transformer and an illumination correction transformer.
Rethinking Temperature in Graph Contrastive Learning
In this paper, we investigate and discuss what a good representation should be for a general loss (InfoNCE) in graph contrastive learning.
ES-Net: Erasing Salient Parts to Learn More in Re-Identification
In this paper, we propose a novel network, Erasing-Salient Net (ES-Net), to learn comprehensive features by erasing the salient areas in an image.
Complementary Pseudo Labels For Unsupervised Domain Adaptation On Person Re-identification
In this paper, to complement these low recall neighbor pseudo labels, we propose a joint learning framework to learn better feature embeddings via high precision neighbor pseudo labels and high recall group pseudo labels.
High-Performance Discriminative Tracking With Transformers
End-to-end discriminative trackers improve the state of the art significantly, yet the improvement in robustness and efficiency is restricted by the conventional discriminative model, i. e., least-squares based regression.
STEAM: Self-Supervised Taxonomy Expansion with Mini-Paths
We propose a self-supervised taxonomy expansion model named STEAM, which leverages natural supervision in the existing taxonomy for expansion.
Discovering Protagonist of Sentiment with Aspect Reconstructed Capsule Network
In this paper, we aim to improve ATSA by discovering the potential aspect terms of the predicted sentiment polarity when the aspect terms of a test sentence are unknown.
