Search Results for author:
Found 71 papers, 35 papers with code
Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection
Therefore, in this paper, we are the first to jointly perform multi-modal ATE (MATE) and multi-modal ASC (MASC), and we propose a multi-modal joint learning approach with auxiliary cross-modal relation detection for multi-modal aspect-level sentiment analysis (MALSA).
Multi-modal Multi-label Emotion Detection with Modality and Label Dependence
In this paper, we focus on multi-label emotion detection in a multi-modal scenario.
On the Temperature of Machine Learning Systems
We consider that the initial potential energy of a ML system is described by the model's loss functions, and the energy adheres to the principle of minimum potential energy.
SpeechAlign: Aligning Speech Generation to Human Preferences
However, the integration of human feedback to align speech outputs to human preferences is often neglected.
Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers
Non-linear functions are prevalent in Transformers and their lightweight variants, incurring substantial and frequently underestimated hardware costs.
Unleashing Network Potentials for Semantic Scene Completion
Semantic scene completion (SSC) aims to predict complete 3D voxel occupancy and semantics from a single-view RGB-D image, and recent SSC methods commonly adopt multi-modal inputs.
Location-guided Head Pose Estimation for Fisheye Image
Camera with a fisheye or ultra-wide lens covers a wide field of view that cannot be modeled by the perspective projection.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music.
Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection
The growing importance of multi-modal humor detection within affective computing correlates with the expanding influence of short-form video sharing on social media platforms.
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Leveraging the rich linguistic knowledge and strong reasoning abilities of LLMs, our new paradigm can integrate the rich information in N-best candidates to generate a higher-quality translation result.
Boundary and Relation Distillation for Semantic Segmentation
Concurrently, the relation distillation transfers implicit relations from the teacher model to the student model using pixel-level self-relation as a bridge, ensuring that the student's mask has strong target region connectivity.
SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation
It comprises an autoregressive model based on LLM for semantic information modeling and a non-autoregressive model employing flow matching for perceptual information modeling.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the requirements of different applications.
BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels
To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei.
GroundingGPT:Language Enhanced Multi-modal Grounding Model
Beyond capturing global information like other multi-modal models, our proposed model excels at tasks demanding a detailed understanding of local information within the input.
SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems
In this paper, we propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication.
Towards SAMBA: Segment Anything Model for Brain Tumor Segmentation in Sub-Sharan African Populations
This study was conducted on the Brain Tumor Segmentation (BraTS) Challenge Africa (BraTS-Africa) dataset, which provides a valuable resource for addressing challenges specific to resource-limited settings, particularly the African population, and facilitating the development of effective and more generalizable segmentation algorithms.
Bridging the Gap: Generalising State-of-the-Art U-Net Models to Sub-Saharan African Populations
We replicated a framework that secured the 2nd position in the 2022 BraTS competition to investigate the impact of dataset composition on model performance and pursued four distinct approaches through training a model with: 1) BraTS-Africa data only (train_SSA, N=60), 2) BraTS-Adult Glioma data only (train_GLI, N=1251), 3) both datasets together (train_ALL, N=1311), and 4) through further training the train_GLI model with BraTS-Africa data (train_ftSSA).
Physics-Informed Neural Network for Discovering Systems with Unmeasurable States with Application to Lithium-Ion Batteries
In particular, instead of having loss terms from each differential equation, this method embeds the dynamics into a loss function that quantifies the error between observed and predicted system outputs.
SeqXGPT: Sentence-Level AI-Generated Text Detection
Therefore, it is important to build strong AI-generated text (AIGT) detectors.
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models.
Synthetic Instance Segmentation from Semantic Image Segmentation Masks
SISeg does not require training a semantic or/and instance segmentation model and avoids the need for instance-level image annotations.
Improving Reference-based Distinctive Image Captioning with Contrastive Rewards
A recent DIC method proposes to generate distinctive captions by comparing the target image with a set of semantic-similar reference images, i. e., reference-based DIC (Ref-DIC).
The Brain Tumor Segmentation (BraTS) Challenge 2023: Glioma Segmentation in Sub-Saharan Africa Patient Population (BraTS-Africa)
Thus, the BraTS-Africa Challenge provides a unique opportunity to include brain MRI glioma cases from SSA in global efforts through the BraTS Challenge to develop and evaluate computer-aided-diagnostic (CAD) methods for the detection and characterization of glioma in resource-limited settings, where the potential for CAD tools to transform healthcare are more likely.
DUB: Discrete Unit Back-translation for Speech Translation
The key point is to bridge the modality gap between speech and text so that useful MT techniques can be applied to ST.
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT.
Should ChatGPT and Bard Share Revenue with Their Data Providers? A New Business Model for the AI Era
Sharing revenue with data providers using such a scoring system would encourage more data owners to participate in the revenue-sharing program.
Rethinking Boundary Detection in Deep Learning Models for Medical Image Segmentation
Medical image segmentation is a fundamental task in the community of medical image analysis.
Discrepancy-Guided Reconstruction Learning for Image Forgery Detection
In this paper, we propose a novel image forgery detection paradigm for boosting the model learning capacity on both forgery-sensitive and genuine compact visual patterns.
Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation
Specifically, a flexible context aggregation module is proposed to capture the global object context in different granular spaces.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Boosting Convolution with Efficient MLP-Permutation for Volumetric Medical Image Segmentation
Recently, the advent of vision Transformer (ViT) has brought substantial advancements in 3D dataset benchmarks, particularly in 3D volumetric medical image segmentation (Vol-MedSeg).
Semantic Scene Completion with Cleaner Self
SSC is a well-known ill-posed problem as the prediction model has to "imagine" what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF).
Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual Constraints
In this paper, we propose a novel framework, TransPro, that translates 3D Optical Coherence Tomography (OCT) images into exclusive 3D OCTA images using an image translation pattern.
Deep-Learning Tool for Early Identifying Non-Traumatic Intracranial Hemorrhage Etiology based on CT Scan
Conclusions: The proposed deep learning algorithms can be an effective tool for early identification of hemorrhage etiologies based on NCCT scans.
Protocol selection for second-order consensus against disturbance
Noticing that both the absolute and relative velocity protocols can solve the second-order consensus of multi-agent systems, this paper aims to investigate which of the above two protocols has better anti-disturbance capability, in which the anti-disturbance capability is measured by the L2 gain from the disturbance to the consensus error.
Centralized Feature Pyramid for Object Detection
To address this problem, in this paper, we propose a Centralized Feature Pyramid (CFP) for object detection, which is based on a globally explicit centralized feature regulation.
Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions
Over the past few years, the rapid development of deep learning technologies for computer vision has significantly improved the performance of medical image segmentation (MedISeg).
Graph Reasoning Transformer for Image Parsing
In this paper, we propose a novel Graph Reasoning Transformer (GReaT) for image parsing to enable image patches to interact following a relation reasoning pattern.
Rethinking the Reference-based Distinctive Image Captioning
Unfortunately, reference images used by existing Ref-DIC works are easy to distinguish: these reference images only resemble the target image at scene-level and have few common objects, such that a Ref-DIC model can trivially generate distinctive captions even without considering the reference images.
FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation
To relax this assumption, in this work, we propose a label-agnostic unified federated learning framework, named FedMix, for medical image segmentation based on mixed image labels.
Learning to Reduce Information Bottleneck for Object Detection in Aerial Images
In this letter, we first underline the importance of the neck network in object detection from the perspective of information bottleneck.
The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights
Combinatorial optimization is a well-established area in operations research and computer science.
FaceAtlasAR: Atlas of Facial Acupuncture Points in Augmented Reality
Acupuncture is a technique in which practitioners stimulate specific points on the body.
Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal
To enhance the robustness of the system and reduce data collecting efforts, we design a data augmentation framework for mmWave signals based on correlations between signal patterns and gesture variations.
Cell-Level State of Charge Estimation for Battery Packs Under Minimal Sensing
This manuscript presents an algorithm for individual Lithium-ion (Li-ion) battery cell state of charge (SOC) estimation in a large-scale battery pack under minimal sensing, where only pack-level voltage and current are measured.
More than Text: Multi-modal Chinese Word Segmentation
Chinese word segmentation (CWS) is undoubtedly an important basic task in natural language processing.
Region-Aware Network: Model Human's Top-Down Visual Perception Mechanism for Crowd Counting
More specifically, we scan the whole input images and its priority maps in the form of column vector to obtain a relevance matrix estimating their similarity.
Learning Calibrated-Guidance for Object Detection in Aerial Images
Specifically, for a given set of feature maps, CG first computes the feature similarity between each channel and the remaining channels as the intermediary calibration guidance.
Machine Learning based Malicious Payload Identification in Software-Defined Networking
In this paper, a novel OpenFlow-enabled deep packet inspection (OFDPI) approach is proposed based on the SDN paradigm to provide adaptive and efficient packet inspection.
Networking and Internet Architecture
Deep image prior for undersampling high-speed photoacoustic microscopy
Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound.
Causal Intervention for Weakly-Supervised Semantic Segmentation
We present a causal inference framework to improve Weakly-Supervised Semantic Segmentation (WSSS).
Ranked #36 on
Weakly-Supervised Semantic Segmentation
on COCO 2014 val
Dual-SLAM: A framework for robust single camera navigation
As local pose estimation is ill-conditioned, local pose estimation failures happen regularly, making the overall SLAM system brittle.
Mask Detection and Breath Monitoring from Speech: on Data Augmentation, Feature Representation and Modeling
This paper introduces our approaches for the Mask and Breathing Sub-Challenge in the Interspeech COMPARE Challenge 2020.
Feature Pyramid Transformer
Yet, the non-local spatial interactions are not across scales, and thus they fail to capture the non-local contexts of objects (or parts) residing in different scales.
Reconstructing undersampled photoacoustic microscopy images using deep learning
One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed.
A multi-label classification method using a hierarchical and transparent representation for paper-reviewer recommendation
Paper-reviewer recommendation task is of significant academic importance for conference chairs and journal editors.
Direct Quantification for Coronary Artery Stenosis Using Multiview Learning
The proposed DMQCA model consists of a multiview module with two attention mechanisms, a key-frame module, and a regression module, to achieve direct accurate multiple-index estimation.
Modeling both context- and speaker-sensitive dependence for emotion detection in multi-speaker conversations
On the one hand, our approach represents each utterance and each speaker as a node.
Ranked #56 on
Emotion Recognition in Conversation
on MELD
Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds
With multiple crowd gatherings of millions of people every year in events ranging from pilgrimages to protests, concerts to marathons, and festivals to funerals; visual crowd analysis is emerging as a new frontier in computer vision.
Ranked #12 on
Crowd Counting
on UCF-QNRF
Video Fill In the Blank using LR/RL LSTMs with Spatial-Temporal Attentions
Since the source sentence is broken into two fragments: the sentence's left fragment (before the blank) and the sentence's right fragment (after the blank), traditional Recurrent Neural Networks cannot encode this structure accurately because of many possible variations of the missing word in terms of the location and type of the word in the source sentence.
ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information
To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the convolutional architecture, and proposes regions of objects of interest (ROOBI).
Unsupervised Action Proposal Ranking through Proposal Recombination
Given the action proposals in a video, the goal of the proposed work is to generate a few better action proposals that are ranked properly.
User Classification with Multiple Textual Perspectives
Textual information is of critical importance for automatic user classification in social media.
Two-View Label Propagation to Semi-supervised Reader Emotion Classification
In this paper, we propose a two-view label propagation approach to semi-supervised reader emotion classification by exploiting two views, namely source text and response text in a label propagation algorithm.
Video Fill in the Blank with Merging LSTMs
In the experiments, we have demonstrated the superior performance of the proposed method on the challenging "Movie Fill-in-the-Blank" dataset.
Local feature hierarchy for face recognition across pose and illumination
Accordingly we propose an end-to-end face recognition method to deal with pose and illumination simultaneously based on convolutional networks where the discriminative nonlinear features that are invariant to pose and illumination are extracted.
A Framework for Human Pose Estimation in Videos
A sequence of the best poses is inferred from the abstract body part tracklets through the tree-based optimization.
Robust Scene Text Recognition Using Sparse Coding based Features
For character detection, we use the HSC features instead of using the Histograms of Oriented Gradients (HOG) features.
Human Pose Estimation in Videos
Using the idea of `Association', the optimal tracklets are generated for each abstract body part, in order to enforce the spatiotemporal constraints between body parts in adjacent frames.
Face Verification Using Boosted Cross-Image Features
In contrast, we propose to extract cross-image features, i. e. features across the pair of images, which, as we demonstrate, is more discriminative to the similarity and the dissimilarity of faces.
Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions
The proposed approach has several contributions: First, a novel layered Directed Acyclic Graph (DAG) based framework is presented for detection and segmentation of the primary object in video.
