Search Results for author:
Found 33 papers, 24 papers with code
Large Multimodal Agents: A Survey
In this paper, we conduct a systematic review of LLM-driven multimodal agents, which we refer to as large multimodal agents ( LMAs for short).
ALLaVA: Harnessing GPT4V-synthesized Data for A Lite Vision-Language Model
Recent advancements in Large Vision-Language Models (LVLMs) have enabled processing of multimodal inputs in language models but require significant computational resources for deployment, especially in edge devices.
CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation
However, developing FMs that can accurately interpret CXRs is challenging due to the (1) limited availability of large-scale vision-language datasets in the medical image domain, (2) lack of vision and language encoders that can capture the complexities of medical data, and (3) absence of evaluation frameworks for benchmarking the abilities of FMs on CXR interpretation.
MLLM-Bench, Evaluating Multi-modal LLMs using GPT-4V
In the pursuit of Artificial General Intelligence (AGI), the integration of vision in language models has marked a significant milestone.
Exploiting Low-confidence Pseudo-labels for Source-free Object Detection
The LPU module consists of Proposal Soft Training (PST) and Local Spatial Contrastive Learning (LSCL).
AceGPT, Localizing Large Language Models in Arabic
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models.
CMB: A Comprehensive Medical Benchmark in Chinese
We hope this benchmark provide first-hand experience in existing LLMs for medicine and also facilitate the widespread adoption and enhancement of medical LLMs within China.
Advancing Visual Grounding with Scene Knowledge: Benchmark and Method
Therefore, in this paper, we propose a novel benchmark of \underline{S}cene \underline{K}nowledge-guided \underline{V}isual \underline{G}rounding (SK-VG), where the image content and referring expressions are not sufficient to ground the target objects, forcing the models to have a reasoning ability on the long-form scene knowledge.
Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation
Parameter Efficient Tuning (PET) has gained attention for reducing the number of parameters while maintaining performance and providing better hardware resource savings, but few studies investigate dense prediction tasks and interaction between modalities.
Ranked #2 on
Referring Expression Segmentation
on RefCOCO
On the Difference of BERT-style and CLIP-style Text Encoders
Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e. g., BERT, one of the representative models.
HuatuoGPT, towards Taming Language Model to Be a Doctor
Experimental results demonstrate that HuatuoGPT achieves state-of-the-art results in performing medical consultation among open-source LLMs in GPT-4 evaluation, human evaluation, and medical benchmark datasets.
Phoenix: Democratizing ChatGPT across Languages
This paper presents our efforts to democratize ChatGPT across language.
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
Medical vision-and-language pre-training (Med-VLP) has shown promising improvements on many downstream medical tasks owing to its applicability to extracting generic representations from medical images and texts.
GIPA: A General Information Propagation Algorithm for Graph Learning
Specifically, the bit-wise correlation calculates the element-wise attention weight through a multi-layer perceptron (MLP) based on the dense representations of two nodes and their edge; The feature-wise correlation is based on the one-hot representations of node attribute features for feature selection.
Ranked #1 on
Node Property Prediction
on ogbn-proteins
Generalizing Multimodal Variational Methods to Sets
Making sense of multiple modalities can yield a more comprehensive description of real-world phenomena.
Toward expanding the scope of radiology report summarization to multiple anatomies and modalities
We then conduct extensive experiments to evaluate the performance of models both within and across modality-anatomy pairs in MIMIC-RRS.
Improving Radiology Summarization with Radiograph and Anatomy Prompts
The impression is crucial for the referring physicians to grasp key information since it is concluded from the findings and reasoning of radiologists.
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training
Besides, we conduct further analysis to better verify the effectiveness of different components of our approach and various settings of pre-training.
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge
Most existing methods mainly contain three elements: uni-modal encoders (i. e., a vision encoder and a language encoder), a multi-modal fusion module, and pretext tasks, with few studies considering the importance of medical domain expert knowledge and explicitly exploiting such knowledge to facilitate Med-VLP.
Cross-modal Memory Networks for Radiology Report Generation
Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later treatments.
Graph Enhanced Contrastive Learning for Radiology Findings Summarization
To address the limitation, we propose a unified framework for exploiting both extra knowledge and the original findings in an integrated way so that the critical information (i. e., key words and their relations) can be extracted in an appropriate way to facilitate impression generation.
Word Graph Guided Summarization for Radiology Findings
In this paper, we propose a novel method for automatic impression generation, where a word graph is constructed from the findings to record the critical words and their relations, then a Word Graph guided Summarization model (WGSum) is designed to generate impressions with the help of the word graph.
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
In this paper, we summarize the recent progress of pre-trained language models in the biomedical domain and their applications in biomedical downstream tasks.
Path-based Deep Network for Candidate Item Matching in Recommenders
PDN utilizes Trigger Net to capture the user's interest in each of his/her interacted item, and Similarity Net to evaluate the similarity between each interacted item and the target item based on these items' profile and CF information.
Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing
Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios.
Generating Radiology Reports via Memory-driven Transformer
Particularly, this is the first work reporting the generation results on MIMIC-CXR to the best of our knowledge.
Attention-Guided Discriminative Region Localization and Label Distribution Learning for Bone Age Assessment
However, training with the global image underutilizes discriminative local information, while providing extra annotations is expensive and subjective.
ESAM: Discriminative Domain Adaptation with Non-Displayed Items to Improve Long-Tail Performance
Most of ranking models are trained only with displayed items (most are hot items), but they are utilized to retrieve items in the entire space which consists of both displayed and non-displayed items (most are long-tail items).
HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation
In particular, our proposed HoMM can perform arbitrary-order moment tensor matching, we show that the first-order HoMM is equivalent to Maximum Mean Discrepancy (MMD) and the second-order HoMM is equivalent to Correlation Alignment (CORAL).
Selective Transfer with Reinforced Transfer Network for Partial Domain Adaptation
However, since the domain shift between source and target domains, only using the deep features for sample selection is defective.
Ranked #6 on
Partial Domain Adaptation
on Office-31
Towards Self-similarity Consistency and Feature Discrimination for Unsupervised Domain Adaptation
Recent advances in unsupervised domain adaptation mainly focus on learning shared representations by global distribution alignment without considering class information across domains.
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation
Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities.
Ro-SOS: Metric Expression Network (MEnet) for Robust Salient Object Segmentation
In this paper, we propose an end-to-end generic salient object segmentation model called Metric Expression Network (MEnet) to deal with saliency detection with the tolerance of distortion.
