Search Results for author:
Found 19 papers, 10 papers with code
What Makes Multimodal In-Context Learning Work?
Large Language Models have demonstrated remarkable performance across various tasks, exhibiting the capacity to swiftly acquire new skills, such as through In-Context Learning (ICL) with minimal demonstration examples.
FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models
The pipeline is as follows: the image is passed to both a captioner model (i. e. BLIP) and a diffusion model (i. e., Stable Diffusion Model) to generate a text description and visual representation, respectively.
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs
The abilities of large language models (LLMs) have recently progressed to unprecedented levels, paving the way to novel applications in a wide variety of areas.
Zero-Shot Refinement of Buildings' Segmentation Models using SAM
For this aim, we present a novel approach to adapt foundation models to address existing models' generalization dropback.
Extending CAM-based XAI methods for Remote Sensing Imagery Segmentation
This paper offers to bridge this gap by adapting the recent XAI classification algorithms and making them usable for muti-class image segmentation, where we mainly focus on buildings' segmentation from high-resolution satellite images.
Empirical Study of PEFT techniques for Winter Wheat Segmentation
This study seeks to bridge this gap by comprehensively exploring the feasibility of cross-area and cross-year out-of-distribution generalization using the State-of-the-Art (SOTA) wheat crop monitoring model.
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Based on our ICL study, (3) we push ICL further and propose new multimodal ICL variants such as; Multitask-ICL, Chain-of-Hindsight-ICL, and Self-Correcting-ICL.
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks
Our model is efficiently pretrained on many tasks, based on task balancing and multimodal curriculum learning.
eP-ALM: Efficient Perceptual Augmentation of Language Models
In this work, we propose to rather direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception.
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
Finally, we validate the generalization of the approach to other tasks (i. e, Food Recognition) and domains with structured text such as the Medical domain on the ROCO dataset.
Ranked #1 on
Cross-Modal Retrieval
on Recipe1M+
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment
Vision and Language Pretraining has become the prevalent approach for tackling multimodal downstream tasks.
Video Coding Using Learned Latent GAN Compression
We leverage the generative capacity of GANs such as StyleGAN to represent and compress a video, including intra and inter compression.
Semantic Unfolding of StyleGAN Latent Space
Generative adversarial networks (GANs) have proven to be surprisingly efficient for image editing by inverting and manipulating the latent code corresponding to an input real image.
Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval
We propose a new retrieval framework, T-Food (Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval) that exploits the interaction between modalities in a novel regularization scheme, while using only unimodal encoders at test time for efficient retrieval.
Ranked #3 on
Cross-Modal Retrieval
on Recipe1M
Buildings Classification using Very High Resolution Satellite Imagery
Buildings classification using satellite images is becoming more important for several applications such as damage assessment, resource allocation, and population estimation.
Sci-Net: Scale Invariant Model for Buildings Segmentation from Aerial Imagery
Buildings' segmentation is a fundamental task in the field of earth observation and aerial imagery analysis.
Learning Perceptual Compression of Facial Video
We leverage the generative capacity of GANs such as StyleGAN to represent and compress each video frame (intra compression), as well as the successive differences between frames (inter compression).
Semantic and Geometric Unfolding of StyleGAN Latent Space
Generative adversarial networks (GANs) have proven to be surprisingly efficient for image editing by inverting and manipulating the latent code corresponding to a natural image.
Synthetic training data generation for deep learning based quality inspection
In order to detect defects, supervised learning is often utilized, but necessitates a large amount of annotated images, which can be costly: collecting, cleaning, and annotating the data is tedious and limits the speed at which a system can be deployed as everything the system must detect needs to be observed first.
