Search Results for author:
Found 80 papers, 50 papers with code
AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception
To address the above challenge, we first introduce a comprehensively annotated Aesthetic Multi-Modality Instruction Tuning (AesMMIT) dataset, which serves as the footstone for building multi-modality aesthetics foundation models.
Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction
Lastly, to further improve the training process, FSS proposes a mesh thickness loss signal for pixel-aligned implicit models.
Towards Open-ended Visual Quality Comparison
Comparative settings (e. g. pairwise choice, listwise ranking) have been adopted by a wide range of subjective studies for image quality assessment (IQA), as it inherently standardizes the evaluation criteria across different observers and offer more clear-cut responses.
MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model
With the evolution of storage and communication protocols, ultra-low bitrate image compression has become a highly demanding topic.
A Benchmark for Multi-modal Foundation Models on Low-level Vision: from Single Images to Pairs
To this end, we design benchmark settings to emulate human language responses related to low-level vision: the low-level visual perception (A1) via visual question answering related to low-level attributes (e. g. clarity, lighting); and the low-level visual description (A2), on evaluating MLLMs for low-level text descriptions.
AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception
An obvious obstacle lies in the absence of a specific benchmark to evaluate the effectiveness of MLLMs on aesthetic perception.
Towards A Better Metric for Text-to-Video Generation
Experiments on the TVGE dataset demonstrate the superiority of the proposed T2VScore on offering a better metric for text-to-video generation.
Q-Refine: A Perceptual Quality Refiner for AI-Generated Image
With the rapid evolution of the Text-to-Image (T2I) model in recent years, their unsatisfactory generation result has become a challenge.
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels
The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents.
Ranked #1 on
Video Quality Assessment
on LIVE-FB LSVQ
Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models
Recent advancements in Multi-modality Large Language Models (MLLMs) have demonstrated remarkable capabilities in complex high-level vision tasks.
Iterative Token Evaluation and Refinement for Real-World Super-Resolution
Distortion removal involves simple HQ token prediction with LQ images, while texture generation uses a discrete diffusion model to iteratively refine the distortion removal output with a token refinement network.
Enhancing Diffusion Models with Text-Encoder Reinforcement Learning
While fine-tuning the U-Net can partially improve performance, it remains suffering from the suboptimal text encoder.
Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models
Multi-modality foundation models, as represented by GPT-4V, have brought a new paradigm for low-level visual perception and understanding tasks, that can respond to a broad range of natural human instructions in a model.
You Only Train Once: A Unified Framework for Both Full-Reference and No-Reference Image Quality Assessment
When our proposed model is independently trained on NR or FR IQA tasks, it outperforms existing models and achieves state-of-the-art performance.
Ranked #1 on
No-Reference Image Quality Assessment
on LIVE
Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
To address this gap, we present Q-Bench, a holistic benchmark crafted to systematically evaluate potential abilities of MLLMs on three realms: low-level visual perception, low-level visual description, and overall visual quality assessment.
IBVC: Interpolation-driven B-frame Video Compression
Learned B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction.
Local Distortion Aware Efficient Transformer Adaptation for Image Quality Assessment
Further, we propose a local distortion extractor to obtain local distortion features from the pretrained CNN and a local distortion injector to inject the local distortion features into ViT.
Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks
On ImageNet, we achieve up to 4. 7% and 4. 6% higher top-1 accuracy compared to other methods for VGG-16 and ResNet-50, respectively.
TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment
Our approach to IQA involves the design of a heuristic coarse-to-fine network (CFANet) that leverages multi-scale features and progressively propagates multi-level semantic information to low-level representations in a top-down manner.
Ranked #11 on
Video Quality Assessment
on MSU SR-QA Dataset
Regression-free Blind Image Quality Assessment with Content-Distortion Consistency
Finally, quality prediction is obtained by aggregating the subjective scores of the retrieved instances.
Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation
To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans.
You Can Mask More For Extremely Low-Bitrate Image Compression
Extensive experiments have demonstrated that our approach outperforms recent state-of-the-art methods in R-D performance, visual quality, and downstream applications, at very low bitrates.
GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment
Model-based 3DQA methods extract features directly from the 3D models, which are characterized by their high degree of complexity.
AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
With the rapid advancements of the text-to-image generative model, AI-generated images (AGIs) have been widely applied to entertainment, education, social media, etc.
Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach
Though subjective studies have collected overall quality scores for these videos, how the abstract quality scores relate with specific factors is still obscure, hindering VQA methods from more concrete quality evaluations (e. g. sharpness of a video).
GCFAgg: Global and Cross-view Feature Aggregation for Multi-view Clustering
However, most existing deep clustering methods learn consensus representation or view-specific representations from multiple views via view-wise aggregation way, where they ignore structure relationship of all samples.
Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment
The proliferation of videos collected during in-the-wild natural settings has pushed the development of effective Video Quality Assessment (VQA) methodologies.
Perceptual Quality Assessment of Face Video Compression: A Benchmark and An Effective Method
In this paper, we introduce the large-scale Compressed Face Video Quality Assessment (CFVQA) database, which is the first attempt to systematically understand the perceptual quality and diversified compression distortions in face videos.
Blind Multimodal Quality Assessment of Low-light Images
Qualitative and quantitative results on Dark-4K show that BMQA achieves superior performance to existing BIQA approaches as long as a pre-trained model is provided to generate text description.
The First Comprehensive Dataset with Multiple Distortion Types for Visual Just-Noticeable Differences
To benefit JND modeling, this work establishes a generalized JND dataset with a coarse-to-fine JND selection, which contains 106 source images and 1, 642 JND maps, covering 25 distortion types.
MetaGrad: Adaptive Gradient Quantization with Hypernetworks
A popular track of network compression approach is Quantization aware Training (QAT), which accelerates the forward pass during the neural network training and inference.
Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterion
Recent learning-based video quality assessment (VQA) algorithms are expensive to implement due to the cost of data collection of human quality opinions, and are less robust across various scenarios due to the biases of these opinions.
JND-Based Perceptual Optimization For Learned Image Compression
After combining them together, we can better assign the distortion in the compressed image with the guidance of JND to preserve the high perceptual quality.
Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation
In this paper, we propose the first simulation model to reveal facial pore changes after using skincare products.
Lightweight Salient Object Detection in Optical Remote-Sensing Images via Semantic Matching and Edge Alignment
Then, semantic kernels are used to activate salient object locations in two groups of high-level features through dynamic convolution operations in DSMM.
Learning Detail-Structure Alternative Optimization for Blind Super-Resolution
Recent blind SR methods suggest to reconstruct SR images relying on blur kernel estimation.
Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution
In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model.
IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction
We propose IntegratedPIFu, a new pixel aligned implicit model that builds on the foundation set by PIFuHD.
DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator
ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models.
Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives
In light of this, we propose the Disentangled Objective Video Quality Evaluator (DOVER) to learn the quality of UGC videos based on the two perspectives.
Ranked #1 on
Video Quality Assessment
on LIVE-VQC
KSS-ICP: Point Cloud Registration based on Kendall Shape Space
The point cloud representation in KSS is invariant to similarity transformations.
Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment
On the other hand, existing practices, such as resizing and cropping, will change the quality of original videos due to the loss of details and contents, and are therefore harmful to quality assessment.
Ranked #2 on
Video Quality Assessment
on KoNViD-1k
(using extra training data)
Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided Resampling
Objective quality assessment of 3D point clouds is essential for the development of immersive multimedia systems in real-world applications.
HVS-Inspired Signal Degradation Network for Just Noticeable Difference Estimation
However, they have a major drawback that the generated JND is assessed in the real-world signal domain instead of in the perceptual domain in the human brain.
Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment
Experiments show that the perceptual representation in the HVS is an effective way of predicting subjective temporal quality, and thus TPQI can, for the first time, achieve comparable performance to the spatial quality metric and be even more effective in assessing videos with large temporal variations.
FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling
Consisting of fragments and FANet, the proposed FrAgment Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and learns effective video-quality-related representations.
Ranked #3 on
Video Quality Assessment
on LIVE-VQC
(using extra training data)
DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment
Based on prominent time-series modeling ability of transformers, we propose a novel and effective transformer-based VQA method to tackle these two issues.
Ranked #5 on
Video Quality Assessment
on KoNViD-1k
Minimum Noticeable Difference based Adversarial Privacy Preserving Image Generation
To achieve this, an adversarial loss is firstly proposed to make the deep learning models attacked by the adversarial images successfully.
Distilling Knowledge from Object Classification to Aesthetics Assessment
To deal with this dilemma, we propose to distill knowledge on semantic patterns for a vast variety of image contents from multiple pre-trained object classification (POC) models to an IAA model.
SelfReformer: Self-Refined Network with Transformer for Salient Object Detection
The global and local contexts significantly contribute to the integrity of predictions in Salient Object Detection (SOD).
Ranked #1 on
Salient Object Detection
on PASCAL-S
Automatic Facial Skin Feature Detection for Everyone
We present an automatic facial skin feature detection method that works across a variety of skin tones and age groups for selfies in the wild.
Adjacent Context Coordination Network for Salient Object Detection in Optical Remote Sensing Images
As the key component of ACCoNet, ACCoM activates the salient regions of output features of the encoder and transmits them to the decoder.
Full RGB Just Noticeable Difference (JND) Modelling
Besides, the JND of the red and blue channels are larger than that of the green one according to the experimental results of the proposed model, which demonstrates that more changes can be tolerated in the red and blue channels, in line with the well-known fact that the human visual system is more sensitive to the green channel in comparison with the red and blue ones.
Lightweight Salient Object Detection in Optical Remote Sensing Images via Feature Correlation
Then, following the coarse-to-fine strategy, we generate an initial coarse saliency map from high-level semantic features in a Correlation Module (CorrM).
Auto-Weighted Layer Representation Based View Synthesis Distortion Estimation for 3-D Video Coding
Experimental results show that the VSD can be accurately estimated with the weights learnt by the nonlinear mapping function once its associated S-VSDs are available.
A New Image Codec Paradigm for Human and Machine Uses
Meanwhile, an image predictor is designed and trained to achieve the general-quality image reconstruction with the 16-bit gray-scale profile and signal features.
Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images
In this paper, we propose a novel Multi-Content Complementation Network (MCCNet) to explore the complementarity of multiple content for RSI-SOD.
Towards Top-Down Just Noticeable Difference Estimation of Natural Images
Instead of explicitly formulating and fusing different masking effects in a bottom-up way, the proposed JND estimation model dedicates to first predicting a critical perceptual lossless (CPL) counterpart of the original image and then calculating the difference map between the original image and the predicted CPL image as the JND map.
Low Resolution Information Also Matters: Learning Multi-Resolution Representations for Person Re-Identification
As a prevailing task in video surveillance and forensics field, person re-identification (re-ID) aims to match person images captured from non-overlapped cameras.
CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes
Then, we design a two-level perturbation fusion strategy to alleviate the conflict between the adversarial watermarks generated by different facial images and models.
Voxel Structure-based Mesh Reconstruction from a 3D Point Cloud
Mesh reconstruction from a 3D point cloud is an important topic in the fields of computer graphic, computer vision, and multimedia analysis.
Advanced Geometry Surface Coding for Dynamic Point Cloud Compression
In video-based dynamic point cloud compression (V-PCC), 3D point clouds are projected onto 2D images for compressing with the existing video codecs.
Just Noticeable Difference for Deep Machine Vision
Experimental results on image classification demonstrate that we successfully find the JND for deep machine vision.
Collaborative Intelligence: Challenges and Opportunities
This paper presents an overview of the emerging area of collaborative intelligence (CI).
Progressive Self-Guided Loss for Salient Object Detection
We present a simple yet effective progressive self-guided loss function to facilitate deep learning-based salient object detection (SOD) in images.
Adversarial Exposure Attack on Diabetic Retinopathy Imagery
In this paper, we study this problem from the viewpoint of adversarial attack and identify a totally new task, i. e., adversarial exposure attack generating adversarial images by tuning image exposure to mislead the DNNs with significantly high transferability.
Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack
To this end, we initiate the very first attempt to study this problem from the perspective of adversarial attack and propose the adversarial denoise attack.
GSTO: Gated Scale-Transfer Operation for Multi-Scale Feature Learning in Pixel Labeling
Existing CNN-based methods for pixel labeling heavily depend on multi-scale features to meet the requirements of both semantic comprehension and detail preservation.
Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization
Image-based localization (IBL) aims to estimate the 6DOF camera pose for a given query image.
Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data
Given an image, our quality measure first extracts 17 features through analysis of contrast, sharpness, brightness and more, and then yields a measre of visual quality using a regression module, which is learned with big-data training samples that are much bigger than the size of relevant image datasets.
A Dilated Inception Network for Visual Saliency Prediction
In this work, we proposed an end-to-end dilated inception network (DINet) for visual saliency prediction.
Towards Robust Curve Text Detection with Conditional Spatial Expansion
It is challenging to detect curve texts due to their irregular shapes and varying sizes.
Which Has Better Visual Quality: The Clear Blue Sky or a Blurry Animal?
The proposed method, SFA, is compared with nine representative blur-specific NR-IQA methods, two general-purpose NR-IQA methods, and two extra full-reference IQA methods on Gaussian blur images (with and without Gaussian noise/JPEG compression) and realistic blur images from multiple databases, including LIVE, TID2008, TID2013, MLIVE1, MLIVE2, BID, and CLIVE.
Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing
The recent advances of hardware technology have made the intelligent analysis equipped at the front-end with deep learning more prevailing and practical.
Robustness Analysis of Pedestrian Detectors for Surveillance
In this study, we conduct the research on the robustness of pedestrian detection algorithms to video quality degradation.
Learning Markov Clustering Networks for Scene Text Detection
MCN predicts instance-level bounding boxes by firstly converting an image into a Stochastic Flow Graph (SFG) and then performing Markov Clustering on this graph.
Review of Visual Saliency Detection with Comprehensive Information
With the acquisition technology development, more comprehensive information, such as depth cue, inter-image correspondence, or temporal relationship, is available to extend image saliency detection to RGBD saliency detection, co-saliency detection, or video saliency detection.
An Iterative Co-Saliency Framework for RGBD Images
In this paper, we propose an iterative RGBD co-saliency framework, which utilizes the existing single saliency maps as the initialization, and generates the final RGBD cosaliency map by using a refinement-cycle model.
Image Quality Assessment Guided Deep Neural Networks Training
For many computer vision problems, the deep neural networks are trained and validated based on the assumption that the input images are pristine (i. e., artifact-free).
