Search Results for author:
Found 54 papers, 22 papers with code
Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentation
Recent developments underscore the potential of textual information in enhancing learning models for a deeper understanding of medical visual semantics.
Mechanisms of non-factual hallucinations in language models
Our study investigates the mechanistic causes of hallucination, specifically non-factual ones where the LM incorrectly predicts object attributes in response to subject-relation queries.
depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers
PyTorch \texttt{2. x} introduces a compiler designed to accelerate deep learning programs.
Predicting Learning Performance with Large Language Models: A Study in Adult Literacy
This research is motivated by the potential of LLMs to predict learning performance based on its inherent reasoning and computational capabilities.
Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation
In this paper, we propose a method to evaluate the response preference by using the output probabilities of response pairs under contrastive prompt pairs, which could achieve better performance on LLaMA2-7B and LLaMA2-13B compared to RLAIF.
Recommendation Fairness in Social Networks Over Time
We further study how interventions on network properties influence fairness by examining counterfactual scenarios with alternative evolution outcomes and differing network properties.
3DG: A Framework for Using Generative AI for Handling Sparse Learner Performance Data From Intelligent Tutoring Systems
Learning performance data (e. g., quiz scores and attempts) is significant for understanding learner engagement and knowledge mastery level.
Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation
We investigate this approach under two different settings: one where the policy model is smaller and is paired with a more powerful critic model, and another where a single language model fulfills both roles.
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals.
Exploring Recommendation Capabilities of GPT-4V(ision): A Preliminary Case Study
Large Multimodal Models (LMMs) have demonstrated impressive performance across various vision and language tasks, yet their potential applications in recommendation tasks with visual assistance remain unexplored.
Successor Features for Efficient Multisubject Controlled Text Generation
While large language models (LLMs) have achieved impressive performance in generating fluent and realistic text, controlling the generated text so that it exhibits properties such as safety, factuality, and non-toxicity remains challenging.
Video Referring Expression Comprehension via Transformer with Content-conditioned Query
Video Referring Expression Comprehension (REC) aims to localize a target object in videos based on the queried natural language.
Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model
In the CPT and SFT phases, Qilin-Med achieved 38. 4% and 40. 0% accuracy on the CMExam test set, respectively.
VeCLIP: Improving CLIP Training via Visual-enriched Captions
For example, VeCLIP achieves up to +25. 2% gain in COCO and Flickr30k retrieval tasks under the 12M setting.
G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theory
Due to two annoying issues in video grounding: (1) the co-existence of some visual entities in both ground truth and other moments, \ie semantic overlapping; (2) only a few moments in the video are annotated, \ie sparse annotation dilemma, vanilla contrastive learning is unable to model the correlations between temporally distant moments and learned inconsistent video representations.
Improving Retrieval-Augmented Large Language Models via Data Importance Learning
There are exponentially many terms in the multilinear extension, and one key contribution of this paper is a polynomial time algorithm that computes exactly, given a retrieval-augmented model with an additive utility function and a validation set, the data importance of data points in the retrieval corpus using the multilinear extension of the model's utility function.
Improving Reference-based Distinctive Image Captioning with Contrastive Rewards
A recent DIC method proposes to generate distinctive captions by comparing the target image with a set of semantic-similar reference images, i. e., reference-based DIC (Ref-DIC).
VISION Datasets: A Benchmark for Vision-based InduStrial InspectiON
We introduce the VISION Datasets, a diverse collection of 14 industrial inspection datasets, uniquely poised to meet these challenges.
Efficient ConvBN Blocks for Transfer Learning and Beyond
Subsequently, we propose a novel Tune mode to bridge the gap between Eval mode and Deploy mode.
Systematic Rectification of Language Models via Dead-end Analysis
Other methods rely on rule-based or prompt-based token elimination, which are limited as they dismiss future tokens and the overall meaning of the complete discourse.
RGI: robust GAN-inversion for mask-free image inpainting and unsupervised pixel-wise anomaly detection
In this paper, we propose a Robust GAN-inversion (RGI) method with a provable robustness guarantee to achieve image restoration under unknown \textit{gross} corruptions, where a small fraction of pixels are completely corrupted.
Learning with Rejection for Abstractive Text Summarization
State-of-the-art abstractive summarization systems frequently hallucinate content that is not supported by the source document, mainly due to noise in the training dataset.
Exploiting Auxiliary Caption for Video Grounding
Video grounding aims to locate a moment of interest matching the given query sentence from an untrimmed video.
Iterative Proposal Refinement for Weakly-Supervised Video Grounding
Weakly-Supervised Video Grounding (WSVG) aims to localize events of interest in untrimmed videos with only video-level annotations.
Video Referring Expression Comprehension via Transformer with Content-aware Query
To this end, we introduce two new datasets (i. e., VID-Entity and VidSTG-Entity) by augmenting the VIDSentence and VidSTG datasets with the explicitly referred words in the whole sentence, respectively.
Adversarial Inter-Group Link Injection Degrades the Fairness of Graph Neural Networks
We hope our findings raise awareness about this issue in our community and lay a foundation for the future development of GNN models that are more robust to such attacks.
Latent Heterogeneous Graph Network for Incomplete Multi-View Learning
Multi-view learning has progressed rapidly in recent years.
Correspondence Matters for Video Referring Expression Comprehension
Extensive experiments demonstrate that our DCNet achieves state-of-the-art performance on both video and image REC benchmarks.
LocVTP: Video-Text Pre-training for Temporal Localization
To further enhance the temporal reasoning ability of the learned feature, we propose a context projection head and a temporal aware contrastive loss to perceive the contextual relationships.
A Survey on Neural Abstractive Summarization Methods and Factual Consistency of Summarization
Automatic summarization is the process of shortening a set of textual data computationally, to create a subset (a summary) that represents the most important pieces of information in the original text.
Jacobian Norm for Unsupervised Source-Free Domain Adaptation
Unsupervised Source (data) Free domain adaptation (USFDA) aims to transfer knowledge from a well-trained source model to a related but unlabeled target domain.
PAEDID: Patch Autoencoder Based Deep Image Decomposition For Pixel-level Defective Region Segmentation
To combine the best of both worlds, we present an unsupervised patch autoencoder based deep image decomposition (PAEDID) method for defective region segmentation.
Unsupervised Pre-training for Temporal Action Localization Tasks
These pre-trained models can be sub-optimal for temporal localization tasks due to the inherent discrepancy between video-level classification and clip-level localization.
Synthetic Defect Generation for Display Front-of-Screen Quality Inspection: A Survey
Display front-of-screen (FOS) quality inspection is essential for the mass production of displays in the manufacturing process.
Information Gain Propagation: a new way to Graph Active Learning with Soft Labels
Graph Neural Networks (GNNs) have achieved great success in various tasks, but their performance highly relies on a large number of labeled nodes, which typically requires considerable human effort.
Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
On active learning task, our method achieves 97. 0% Top-1 Accuracy on CIFAR10 with 0. 1% annotated data, and 83. 9% Top-1 Accuracy on CIFAR100 with 10% annotated data.
RIM: Reliable Influence-based Active Learning on Graphs
Message passing is the core of most graph models such as Graph Convolutional Network (GCN) and Label Propagation (LP), which usually require a large number of clean labeled data to smooth out the neighborhood over the graph.
On Pursuit of Designing Multi-modal Transformer for Video Grounding
Almost all existing video grounding methods fall into two frameworks: 1) Top-down model: It predefines a set of segment candidates and then conducts segment classification and regression.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
State-of-the-art abstractive summarization systems often generate \emph{hallucinations}; i. e., content that is not directly inferable from the source text.
Abstractive Text Summarization
Reinforcement Learning (RL)
+1
Deep Motion Prior for Weakly-Supervised Temporal Action Localization
In this paper, we analyze that the motion cues behind the optical flow features are complementary informative.
Optical Flow Estimation
Weakly-supervised Temporal Action Localization
+1
UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
All You Need is a Second Look: Towards Arbitrary-Shaped Text Detection
Compared to the traditional single-stage segmentation network, our NASK conducts the detection in a coarse-to-fine manner with the first stage segmentation spotting the rectangle text proposals and the second one retrieving compact representations.
BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
While single-shot quantized neural architecture search enjoys flexibility in both model architecture and quantization policy, the combined search space comes with many challenges, including instability when training the weight-sharing supernet and difficulty in navigating the exponentially growing search space.
Hardware Aware Neural Architecture Search
Model Optimization
+2
Video Frame Interpolation via Structure-Motion based Iterative Fusion
Inspired by the observation that audiences have different visual preferences on foreground and background objects, we for the first time propose to use saliency masks in the evaluation processes of the task of video frame interpolation.
RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection
Upon the frame, an Interaction Intensifier Module and a Correlation Parsing Module are carefully designed, where: a) interactive semantics from humans can be exploited and passed to objects to intensify interactions, b) interactive correlations among humans, objects and interactions are integrated to promote predictions.
CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning
In this paper, we argue that learning by comparing helps identify these hard snippets and we propose to utilize snippet Contrastive learning to Localize Actions, CoLA for short.
Quantum error-correcting codes from matrix-product codes related to quasi-orthogonal and quasi-unitary matrices
The construction of matrix-product codes with certain self-orthogonality over finite fields is an effective way to obtain good $q$-ary quantum codes of large length.
Information Theory Information Theory Quantum Physics
Factual Error Correction for Abstractive Summarization Models
Experimental results show that our model is able to correct factual errors in summaries generated by other neural summarization models and outperforms previous models on factual consistency evaluation on the CNN/DailyMail dataset.
TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion
Our analysis also reveals important sources of variability both within and across TKG datasets, and we introduce several simple but strong baselines that outperform the prior state of the art in certain settings.
An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department
In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time.
Task-agnostic Temporally Consistent Facial Video Editing
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
All you need is a second look: Towards Tighter Arbitrary shape text detection
Specifically, \textit{NASK} consists of a Text Instance Segmentation network namely \textit{TIS} (\(1^{st}\) stage), a Text RoI Pooling module and a Fiducial pOint eXpression module termed as \textit{FOX} (\(2^{nd}\) stage).
Unsupervised Domain Adaptation Through Transferring both the Source-Knowledge and Target-Relatedness Simultaneously
Unsupervised domain adaptation (UDA) is an emerging research topic in the field of machine learning and pattern recognition, which aims to help the learning of unlabeled target domain by transferring knowledge from the source domain.
Referring Expression Generation Using Entity Profiles
Referring Expression Generation (REG) is the task of generating contextually appropriate references to entities.
