Search Results for author:
Found 26 papers, 15 papers with code
Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
In this study, we aim to shed light on this issue by revealing that masked auto-encoder (MAE) pre-training with enhanced decoding significantly improves the term coverage of input tokens in dense representations, compared to vanilla BERT checkpoints.
MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models
Experiments reveal that models incorporating the proposed MiLe Loss can gain consistent performance improvement on downstream benchmarks.
HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus
ChatGPT has gained significant interest due to its impressive performance, but people are increasingly concerned about its potential risks, particularly around the detection of AI-generated content (AIGC), which is often difficult for untrained humans to identify.
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval
Concretely, we leverage the capabilities of LLMs for document expansion, i. e. query generation, and effectively transfer expanded knowledge to retrievers using pre-training strategies tailored for passage retrieval.
Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems
Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history.
Ranked #1 on
Conversational Response Selection
on E-commerce
PUNR: Pre-training with User Behavior Modeling for News Recommendation
Firstly, we introduce the user behavior masking pre-training task to recover the masked user behaviors based on their contextual behaviors.
CoT-MoTE: Exploring ConTextual Masked Auto-Encoder Pre-training with Mixture-of-Textual-Experts for Passage Retrieval
Siamese or fully separated dual-encoders are often adopted as basic retrieval architecture in the pre-training and fine-tuning stages for encoding queries and passages into their latent embedding spaces.
CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrieval
As an effective representation bottleneck pretraining technique, the contextual masked auto-encoder utilizes contextual embedding to assist in the reconstruction of passages.
Query-as-context Pre-training for Dense Passage Retrieval
Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training.
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval
Sparse sampling is also likely to miss important frames corresponding to some text portions, resulting in textual redundancy.
InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings
Contrastive learning has been extensively studied in sentence embedding learning, which assumes that the embeddings of different views of the same sentence are closer.
Pathway to Future Symbiotic Creativity
This report presents a comprehensive view of our vision on the development path of the human-machine symbiotic art creation.
ConTextual Masked Auto-Encoder for Dense Passage Retrieval
Dense passage retrieval aims to retrieve the relevant passages of a query from a large corpus based on dense representations (i. e., vectors) of the query and the passages.
Stacked Autoencoder Based Multi-Omics Data Integration for Cancer Survival Prediction
In the cancer survival prediction for TCGA cases, SAEsurv-net addresses the curse of dimensionality with a two-stage dimensionality reduction strategy and handles multi-omics heterogeneity with a stacked autoencoder model.
Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks
Before entering the neural network, a token is generally converted to the corresponding one-hot representation, which is a discrete distribution of the vocabulary.
DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings
To tackle that, we propose an effective knowledge distillation framework for contrastive sentence embeddings, termed DistilCSE.
TransAug: Translate as Augmentation for Sentence Embeddings
In this paper, we present TransAug (Translate as Augmentation), which provide the first exploration of utilizing translated sentence pairs as data augmentation for text, and introduce a two-stage paradigm to advances the state-of-the-art sentence embeddings.
Smoothed Contrastive Learning for Unsupervised Sentence Embedding
Contrastive learning has been gradually applied to learn high-quality unsupervised sentence embedding.
ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding
Unsup-SimCSE takes dropout as a minimal data augmentation method, and passes the same input sentence to a pre-trained Transformer encoder (with dropout turned on) twice to obtain the two corresponding embeddings to build a positive pair.
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
As an alternative, we propose a new method for BERT distillation, i. e., asking the teacher to generate smoothed word ids, rather than labels, for teaching the student model in knowledge distillation.
Data Augmentation for Copy-Mechanism in Dialogue State Tracking
While several state-of-the-art approaches to dialogue state tracking (DST) have shown promising performances on several benchmarks, there is still a significant performance gap between seen slot values (i. e., values that occur in both training set and test set) and unseen ones (values that occur in training set but not in test set).
TransSent: Towards Generation of Structured Sentences with Discourse Marker
Automatic and human evaluation results show that TransSent can generate structured sentences with high quality, and has certain scalability in different tasks.
"Mask and Infill" : Applying Masked Language Model to Sentiment Transfer
So we propose a two step approach "Mask and Infill".
Imbalanced Sentiment Classification Enhanced with Discourse Marker
Imbalanced data commonly exists in real world, espacially in sentiment-related corpus, making it difficult to train a classifier to distinguish latent sentiment in text data.
Conditional BERT Contextual Augmentation
BERT demonstrates that a deep bidirectional language model is more powerful than either an unidirectional language model or the shallow concatenation of a forward and backward model.
LUCSS: Language-based User-customized Colourization of Scene Sketches
We introduce LUCSS, a language-based system for interactive col- orization of scene sketches, based on their semantic understanding.
