Search Results for author:
Found 42 papers, 8 papers with code
Enhancing Hyperspectral Images via Diffusion Model and Group-Autoencoder Super-resolution Network
Existing hyperspectral image (HSI) super-resolution (SR) methods struggle to effectively capture the complex spectral-spatial relationships and low-level details, while diffusion models represent a promising generative model known for their exceptional performance in modeling complex relations and learning high and low-level visual features.
Diffusion Model Based Visual Compensation Guidance and Visual Difference Analysis for No-Reference Image Quality Assessment
Firstly, we devise a new diffusion restoration network that leverages the produced enhanced image and noise-containing images, incorporating nonlinear features obtained during the denoising process of the diffusion model, as high-level visual information.
Unlocking the `Why' of Buying: Introducing a New Dataset and Benchmark for Purchase Reason and Post-Purchase Experience
To this end, we introduce an LLM-based approach to generate a dataset that consists of textual explanations of why real users make certain purchase decisions.
On provable privacy vulnerabilities of graph representations
Graph representation learning (GRL) is critical for extracting insights from complex network structures, but it also raises security concerns due to potential privacy vulnerabilities in these representations.
A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective
Vertical Federated Learning (VFL) is a federated learning paradigm where multiple participants, who share the same set of samples but hold different features, jointly train machine learning models.
PRewrite: Prompt Rewriting with Reinforcement Learning
Prompt engineering is critical for the development of LLM-based applications.
Bridging the Preference Gap between Retrievers and LLMs
Large Language Models (LLMs) have demonstrated superior results across a wide range of tasks, and Retrieval-augmented Generation (RAG) is an effective way to enhance the performance by locating relevant information and placing it into the context window of the LLM.
Transfer the linguistic representations from TTS to accent conversion with non-parallel data
This paper introduces a novel non-autoregressive framework for accent conversion that learns accent-agnostic linguistic representations and employs them to convert the accent in the source speech.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Creator Context for Tweet Recommendation
In other words, grounding the interpretation of the tweet in the context of its creator plays an important role in deciphering the true intent and the importance of the tweet.
Privacy-preserving design of graph neural networks with applications to vertical federated learning
The paradigm of vertical federated learning (VFL), where institutions collaboratively train machine learning models via combining each other's local feature or label information, has achieved great success in applications to financial risk management (FRM).
Automated Evaluation of Personalized Text Generation using Large Language Models
Personalized text generation presents a specialized mechanism for delivering content that is specific to a user's personal context.
Learning to Rewrite Prompts for Personalized Text Generation
In this paper, we propose a novel method to automatically revise prompts for personalized text generation.
Teach LLMs to Personalize -- An Approach inspired by Writing Education
Inspired by the practice of writing education, we develop a multistage and multitask framework to teach LLMs for personalized generation.
End-to-End Query Term Weighting
These inferred weights and terms can be used directly by a retrieval system to perform a query search.
Ranked #1 on
Passage Retrieval
on MS MARCO
ShiftNAS: Improving One-shot NAS via Probability Shift
To address the problem of uniform sampling, we propose ShiftNAS, a method that can adjust the sampling probability based on the complexity of subnets.
High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Units
The goal of Automatic Voice Over (AVO) is to generate speech in sync with a silent video given its text script.
LoRAPrune: Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
This is due to their utilization of unstructured pruning on LPMs, impeding the merging of LoRA weights, or their dependence on the gradients of pre-trained weights to guide pruning, which can impose significant memory overhead.
Accented Text-to-Speech Synthesis with Limited Data
Both objective and subjective evaluation results show that the accented TTS front-end fine-tuned with a small accented phonetic lexicon (5k words) effectively handles the phonetic variation of accents, while the accented TTS acoustic model fine-tuned with a limited amount of accented speech data (approximately 3 minutes) effectively improves the prosodic rendering including pitch and duration.
A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image
3D interacting hand pose estimation from a single RGB image is a challenging task, due to serious self-occlusion and inter-occlusion towards hands, confusing similar appearance patterns between 2 hands, ill-posed joint position mapping from 2D to 3D, etc.. To address these, we propose to extend A2J-the state-of-the-art depth-based 3D single hand pose estimation method-to RGB domain under interacting hand condition.
Ranked #7 on
Hand Pose Estimation
on NYU Hands
TTS-Guided Training for Accent Conversion Without Parallel Data
Specifically, a text-to-speech (TTS) system is first pretrained with target-accented speech data.
Bayesian Layer Graph Convolutioanl Network for Hyperspetral Image Classification
In recent years, research on hyperspectral image (HSI) classification has continuous progress on introducing deep network models, and recently the graph convolutional network (GCN) based models have shown impressive performance.
Towards Consistency and Complementarity: A Multiview Graph Information Bottleneck Approach
In this paper, we propose a novel Multiview Variational Graph Information Bottleneck (MVGIB) principle to maximize the agreement for common representations and the disagreement for view-specific representations.
Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy
However, the performance of architecture is limited by the type of operations and prior knowledge.
Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Models
In this work, we carefully select five datasets, including two in-domain datasets and three out-of-domain datasets with different levels of domain shift, and study the generalization of a deep model in a zero-shot setting.
DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding
At the same time, as the deep neural analyzer is learnable, it is expected to be more accurate for signal reconstruction and manipulation, and generalizable from speech to singing.
Across-Task Neural Architecture Search via Meta Learning
In this paper, an across-task neural architecture search (AT-NAS) is proposed to address the problem through combining gradient-based meta-learning with EA-based NAS to learn over the distribution of tasks.
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
The proposed VisualTTS adopts two novel mechanisms that are 1) textual-visual attention, and 2) visual fusion strategy during acoustic decoding, which both contribute to forming accurate alignment between the input text content and lip motion in input lip sequence.
Indoor Localization Using Smartphone Magnetic with Multi-Scale TCN and LSTM
A novel multi-scale temporal convolutional network (TCN) and long short-term memory network (LSTM) based magnetic localization approach is proposed.
RepNAS: Searching for Efficient Re-parameterizing Blocks
However, it is still challenging to search for efficient networks due to the gap between the searched constraint and real inference time exists.
Dynamic Language Models for Continuously Evolving Content
To this end, we both explore two different vocabulary composition methods, as well as propose three sampling methods which help in efficient incremental training for BERT-like models.
Spatially Self-Paced Convolutional Networks for Change Detection in Heterogeneous Images
Change detection in heterogeneous remote sensing images is a challenging problem because it is hard to make a direct comparison in the original observation spaces, and most methods rely on a set of manually labeled samples.
LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding
text, table, image) and propose a novel layout-aware multimodal hierarchical framework, LAMPreT, to model the blocks and the whole document.
Natural Language Understanding with Privacy-Preserving BERT
We investigate the privacy and utility implications of applying dx-privacy, a variant of Local Differential Privacy, to BERT fine-tuning in NLU applications.
Effective Model Compression via Stage-wise Pruning
To overcome the unfull training, a stage-wise pruning(SWP) method is proposed, which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage training.
Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
A hybrid approach, which leverages both semantic (deep neural network-based) and lexical (keyword matching-based) retrieval models, is proposed.
Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion
We consider that there is a common code between speakers for emotional expression in a spoken language, therefore, a speaker-independent mapping between emotional states is possible.
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
In order to better capture sentence level semantic relations within a document, we pre-train the model with a novel masked sentence block language modeling task in addition to the masked word language modeling task used by BERT.
Urban Anomaly Analytics: Description, Detection, and Prediction
Urban anomalies may result in loss of life or property if not handled properly.
Graph Pruning for Model Compression
Different from previous work, we take the node features from a well-trained graph aggregator instead of the hand-craft features, as the states in reinforcement learning.
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Our proposed approach significantly improved the intelligibility (in CER), the MOS, and discrimination ABX scores compared to the official ZeroSpeech 2019 baseline or even the topline.
Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks.
