Search Results for author:
Found 86 papers, 31 papers with code
Grounded Dialogue Generation with Cross-encoding Re-ranker, Grounding Span Prediction, and Passage Dropout
For the response generator, we use grounding span prediction as an auxiliary task to be jointly trained with the main task of response generation.
Few-shot Name Entity Recognition on StackOverflow
StackOverflow, with its vast question repository and limited labeled examples, raise an annotation challenge for us.
AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation
To fill in the gap of the overhead view dataset, this paper constructs and releases an aerial image urban scene graph generation (AUG) dataset.
LPSNet: End-to-End Human Pose and Shape Estimation with Lensless Imaging
Human pose and shape (HPS) estimation with lensless imaging is not only beneficial to privacy protection but also can be used in covert surveillance scenarios due to the small size and simple structure of this device.
Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding
Inspired by the activity-silent and persistent activity mechanisms in human visual perception biology, we design a Unified Static and Dynamic Network (UniSDNet), to learn the semantic association between the video and text/audio queries in a cross-modal environment for efficient video grounding.
Prototyping and Experimental Results for Environment-Aware Millimeter Wave Beam Alignment via Channel Knowledge Map
In this paper, to alleviate the large training overhead in millimeter wave (mmWave) beam alignment, an environment-aware and training-free beam alignment prototype is established based on a typical CKM, termed beam index map (BIM).
Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture
To this end, we present FD4MM, a new paradigm of Frequency Decoupling for Motion Magnification with a Multi-level Isomorphic Architecture to capture multi-level high-frequency details and a stable low-frequency structure (motion field) in video space.
Benchmarking Micro-action Recognition: Dataset, Methods, and Applications
It offers insights into the feelings and intentions of individuals and is important for human-oriented applications such as emotion recognition and psychological assessment.
Ranked #1 on
Micro-Action Recognition
on MA-52
Convincing Rationales for Visual Question Answering Reasoning
Visual Question Answering (VQA) is a challenging task of predicting the answer to a question about the content of an image.
KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching
Different from them, we propose to first align the Ref with the center region (namely the overlapped FoV area) of the LR input by combining global warping and local warping to make the aligned Ref be sharp and consistent.
CLDR: Contrastive Learning Drug Response Models from Natural Language Supervision
At the same time, in order to enhance the continuous representation capability of the numerical text, a common-sense numerical knowledge graph is introduced.
Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Joints
In this work, we introduce Joint2Human, a novel method that leverages 2D diffusion models to generate detailed 3D human geometry directly, ensuring both global structure and local details.
Layered 3D Human Generation via Semantic-Aware Diffusion Model
To keep the generated clothing consistent with the target text, we propose a semantic-confidence strategy for clothing that can eliminate the non-clothing content generated by the model.
R2Human: Real-Time 3D Human Appearance Rendering from a Single Image
In this paper, we propose R2Human, the first approach for real-time inference and rendering of photorealistic 3D human appearance from a single image.
EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer
Then, we introduce a novel dynamic filter that eliminates noise cues and preserves critical features in the motion magnification and amplification generation phases.
SpeechAct: Towards Generating Whole-body Motion from Speech
Then, we regress the motion representation from the audio signal by a translation model employing our contrastive motion learning method.
High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos
To alleviate the ambiguity estimating 3D garments from monocular videos, we design a multi-hypothesis deformation module that learns spatial representations of multiple plausible deformations.
Towards Grouping in Large Scenes with Occlusion-aware Spatio-temporal Transformers
To explore the potential spatio-temporal relationship, we propose spatio-temporal transformers to simultaneously extract trajectory information and fuse inter-person features in a hierarchical manner.
CT-GAT: Cross-Task Generative Adversarial Attack based on Transferability
Neural network models are vulnerable to adversarial examples, and adversarial transferability further increases the risk of adversarial attacks.
MeaeQ: Mount Model Extraction Attacks with Efficient Queries
We study model extraction attacks in natural language processing (NLP) where attackers aim to steal victim models by repeatedly querying the open Application Programming Interfaces (APIs).
Transformer-based Multimodal Change Detection with Multitask Consistency Constraints
We observe that the current change detection methods struggle with the multitask conflicts between semantic and height change detection tasks.
A New Transformation Approach for Uplift Modeling with Binary Outcome
The main drawback of these approaches is that in general it does not use the information in the treatment indicator beyond the construction of the transformed outcome and usually is not efficient.
Zero-shot Learning of Drug Response Prediction for Preclinical Drug Screening
In this paper, we propose a zero-shot learning solution for the DRP task in preclinical drug screening.
Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding
Compared with the general video grounding task, MTVG focuses on meticulous actions and changes on the face.
FusionFormer: A Multi-sensory Fusion in Bird's-Eye-View and Temporal Consistent Transformer for 3D Object Detection
Multi-sensor modal fusion has demonstrated strong advantages in 3D object detection tasks.
Exploiting Diverse Feature for Multimodal Sentiment Analysis
Specifically, we achieve the results of 0. 8492 and 0. 8439 for MuSe-Personalisation in terms of arousal and valence CCC.
Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos
Specifically, the proposed Dual-TL uses a Spatial TokenLearner (S-TL) to explore associations in different facial ROIs, which promises the rPPG prediction far away from noisy ROI disturbances.
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer
First, we employed a sharing feature encoder to project both video and query into a joint feature space before performing cross-modal co-attention (i. e., video-to-query attention and query-to-video attention) to highlight discriminative features in each modality.
Data Augmentation for Human Behavior Analysis in Multi-Person Conversations
In this paper, we present the solution of our team HFUT-VUT for the MultiMediate Grand Challenge 2023 at ACM Multimedia 2023.
FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving
Building a multi-modality multi-task neural network toward accurate and robust performance is a de-facto standard in perception task of autonomous driving.
Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification
In this paper, we briefly introduce the solution of our team HFUT-VUT for the Micros-gesture Classification in the MiGA challenge at IJCAI 2023.
Ranked #1 on
Micro-gesture Recognition
on iMiGUE
Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer
To address these issues, we propose a novel Deformable Motion Modulation (DMM) that utilizes geometric kernel offset with adaptive weight modulation to simultaneously perform feature alignment and style transfer.
ATWM: Defense against adversarial malware based on adversarial training
In order to defend against malware attacks, researchers have proposed many Windows malware detection models based on deep learning.
CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation
In this way, RGB images are not required during inference anymore since the 2D knowledge branch provides 2D information according to the 3D LIDAR input.
Interactive Image Segmentation with Cross-Modality Vision Transformers
Interactive image segmentation aims to segment the target from the background with the manual guidance, which takes as input multimodal data such as images, clicks, scribbles, and bounding boxes.
Imitation with Spatial-Temporal Heatmap: 2nd Place Solution for NuPlan Challenge
This paper presents our 2nd place solution for the NuPlan Challenge 2023.
Physics-Informed Ensemble Representation for Light-Field Image Super-Resolution
Recent learning-based approaches have achieved significant progress in light field (LF) image super-resolution (SR) by exploring convolution-based or transformer-based network structures.
FGAM:Fast Adversarial Malware Generation Method Based on Gradient Sign
Adversarial attacks are to deceive the deep learning model by generating adversarial samples.
SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting
Building end-to-end task bots and maintaining their integration with new functionalities using minimal human efforts is a long-standing challenge in dialog research.
Standardized Benchmark Dataset for Localized Exposure to a Realistic Source at 10$-$90 GHz
To address this issue, in this paper, the limited available data on the incident power density and resultant maximum temperature rise on the skin surface considering various steady-state exposure scenarios at 10$-$90 GHz have been statistically modeled.
HybridFusion: LiDAR and Vision Cross-Source Point Cloud Fusion
First, the point cloud is divided into small patches, and a matching patch set is selected based on global descriptors and spatial distribution, which constitutes the coarse matching process.
Narrator: Towards Natural Control of Human-Scene Interaction Generation via Relationship Reasoning
Also, we model global and local spatial relationships in a 3D scene and a textual description respectively based on the scene graph, and introduce a partlevel action mechanism to represent interactions as atomic body part states.
Angle-Constrained Formation Control under Directed Non-Triangulated Sensing Graphs (Extended Version)
Angle-constrained formation control has attracted much attention from control community due to the advantage that inter-edge angles are invariant under uniform translations, rotations, and scalings of the whole formation.
Causal Inference Based Single-branch Ensemble Trees For Uplift Modeling
In this manuscript (ms), we propose causal inference based single-branch ensemble trees for uplift modeling, namely CIET.
Crowd3D: Towards Hundreds of People Reconstruction from a Single Image
However, existing methods cannot deal with large scenes containing hundreds of people, which encounter the challenges of large number of people, large variations in human scale, and complex spatial distribution.
HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Images
Visual question answering (VQA) is an important and challenging multimodal task in computer vision.
Learning Semantic-Aware Disentangled Representation for Flexible 3D Human Body Editing
3D human body representation learning has received increasing attention in recent years.
Fleet Rebalancing for Expanding Shared e-Mobility Systems: A Multi-agent Deep Reinforcement Learning Approach
This is particularly challenging in the context of expanding systems, because i) the range of the EVs is limited while charging time is typically long, which constrain the viable rebalancing operations; and ii) the EV stations in the system are dynamically changing, i. e., the legitimate targets for rebalancing operations can vary over time.
ITSRN++: Stronger and Better Implicit Transformer Network for Continuous Screen Content Image Super-Resolution
Specifically, we propose a modulation based transformer as the upsampler, which modulates the pixel features in discrete space via a periodic nonlinear function to generate features for continuous pixels.
FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction
Based on FOF, we design the first 30+FPS high-fidelity real-time monocular human reconstruction framework.
HDhuman: High-quality Human Novel-view Rendering from Sparse Views
To this end, we propose HDhuman, which uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network with geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering.
Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition
Emotion recognition is a challenging and actively-studied research area that plays a critical role in emotion-aware human-computer interaction systems.
High-Fidelity Human Avatars From a Single RGB Camera
To cope with the complexity of textures and generate photo-realistic results, we propose a reference-based neural rendering network and exploit a bottom-up sharpening-guided fine-tuning strategy to obtain detailed textures.
AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds
From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements.
Implicit Transformer Network for Screen Content Image Continuous Super-Resolution
Nowadays, there is an explosive growth of screen contents due to the wide application of screen sharing, remote cooperation, and online education.
Large-Scale Hyperspectral Image Clustering Using Contrastive Learning
Specifically, we exploit a symmetric twin neural network comprised of a projection head with a dimensionality of the cluster number to conduct dual contrastive learning from a spectral-spatial augmentation pool.
LSTM-RPA: A Simple but Effective Long Sequence Prediction Algorithm for Music Popularity Prediction
The traditional trend prediction models can better predict the short trend than the long trend.
Real-Time Anchor-Free Single-Stage 3D Detection with IoU-Awareness
Extended from our last year's award-winning model AFDet, we have made a handful of modifications to the base model, to improve the accuracy and at the same time to greatly reduce the latency.
Economic Recession Prediction Using Deep Neural Network
We investigate the effectiveness of different machine learning methodologies in predicting economic cycles.
Out-of-Scope Domain and Intent Classification through Hierarchical Joint Modeling
User queries for a real-world dialog system may sometimes fall outside the scope of the system's capabilities, but appropriate system responses will enable smooth processing throughout the human-computer interaction.
Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing
This paper presents an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances in a domain.
An Accurate and Efficient Large-scale Regression Method through Best Friend Clustering
As the data size in Machine Learning fields grows exponentially, it is inevitable to accelerate the computation by utilizing the ever-growing large number of available cores provided by high-performance computing hardware.
PISE: Person Image Synthesis and Editing with Decoupled GAN
The results of qualitative and quantitative experiments demonstrate the superiority of our model on human pose transfer.
Polycaprolactone/graphite nanoplates composite nanopapers
Nanopapers based on graphene and related materials were recently proposed for application in heat spreader applications.
Materials Science
Human Pose Transfer by Adaptive Hierarchical Deformation
Existing methods cannot effectively utilize the input information, which often fail to preserve the style and shape of hair and clothes.
PoNA: Pose-guided Non-local Attention for Human Pose Transfer
In each block, we propose a pose-guided non-local attention (PoNA) mechanism with a long-range dependency scheme to select more important regions of image features to transfer.
GPS-Net: Graph-based Photometric Stereo Network
For all-pixel operation, we propose the Normal Regression Network to make efficient use of the intra-image spatial information for predicting a surface normal map with rich details.
Constituency Lattice Encoding for Aspect Term Extraction
One of the remaining challenges for aspect term extraction in sentiment analysis resides in the extraction of phrase-level aspect terms, which is non-trivial to determine the boundaries of such terms.
Aspect Term Extraction and Sentiment Classification
Sentence
+1
Cross-MPI: Cross-scale Stereo for Image Super-Resolution using Multiplane Images
Various combinations of cameras enrich computational photography, among which reference-based superresolution (RefSR) plays a critical role in multiscale imaging systems.
Unsupervised Pre-training for Biomedical Question Answering
We explore the suitability of unsupervised representation learning methods on biomedical text -- BioBERT, SciBERT, and BioSentVec -- for biomedical question answering.
A Vertical Federated Learning Method for Interpretable Scorecard and Its Application in Credit Scoring
With the success of big data and artificial intelligence in many fields, the applications of big data driven models are expected in financial risk management especially credit scoring and rating.
Visual-speech Synthesis of Exaggerated Corrective Feedback
To provide more discriminative feedback for the second language (L2) learners to better identify their mispronunciation, we propose a method for exaggerated visual-speech feedback in computer-assisted pronunciation training (CAPT).
Adaptive 3D Face Reconstruction from a Single Image
3D face reconstruction from a single image is a challenging problem, especially under partial occlusions and extreme poses.
A Federated F-score Based Ensemble Model for Automatic Rule Extraction
In this manuscript, we propose a federated F-score based ensemble tree model for automatic rule extraction, namely Fed-FEARE.
Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation
In this paper, we formulate the data augmentation as a conditional generation task: generating a new sentence while preserving the original opinion targets and labels.
4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras
Our method enables a realtime online motion capture system running at 30fps using 5 cameras on a 5-person scene.
Ranked #8 on
3D Multi-Person Pose Estimation
on Shelf
Learning to Reconstruct and Understand Indoor Scenes from Sparse Views
This paper proposes a new method for simultaneous 3D reconstruction and semantic segmentation of indoor scenes.
Accurate 3D Localization for MAV Swarms by UWB and IMU Fusion
Driven by applications like Micro Aerial Vehicles (MAVs), driver-less cars, etc, localization solution has become an active research topic in the past decade.
Robotics
Meta Inverse Reinforcement Learning via Maximum Reward Sharing for Human Motion Analysis
Observing that each demonstrator has an inherent reward for each state and the task-specific behaviors mainly depend on a small number of key states, we propose a meta IRL algorithm that first models the reward function for each task as a distribution conditioned on a baseline reward function shared by all tasks and dependent only on the demonstrator, and then finds the most likely reward function in the distribution that explains the task-specific behaviors.
A Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning
This works handles the inverse reinforcement learning problem in high-dimensional state spaces, which relies on an efficient solution of model-based high-dimensional reinforcement learning problems.
Inverse Reinforcement Learning in Large State Spaces via Function Approximation
We also show that the proposed method can extend many existing methods to high-dimensional state spaces.
Bellman Gradient Iteration for Inverse Reinforcement Learning
We introduce a strategy to flexibly handle different types of actions with two approximations of the Bellman Optimality Equation, and a Bellman Gradient Iteration method to compute the gradient of the Q-value with respect to the reward function.
Clinical Patient Tracking in the Presence of Transient and Permanent Occlusions via Geodesic Feature
This paper develops a method to use RGB-D cameras to track the motions of a human spinal cord injury patient undergoing spinal stimulation and physical rehabilitation.
Robust Non-Rigid Registration with Reweighted Position and Transformation Sparsity
Non-rigid registration is challenging because it is ill-posed with high degrees of freedom and is thus sensitive to noise and outliers.
Inverse Reinforcement Learning with Multi-Relational Chains for Robot-Centered Smart Home
In a robot-centered smart home, the robot observes the home states with its own sensors, and then it can change certain object states according to an operator's commands for remote operations, or imitate the operator's behaviors in the house for autonomous operations.
