Classes Are Not Equal: An Empirical Study on Image Recognition Fairness

1 code implementation • 28 Feb 2024 • Jiequan Cui, Beier Zhu, Xin Wen, Xiaojuan Qi, Bei Yu, Hanwang Zhang

Second, with the proposed concept of Model Prediction Bias, we investigate the origins of problematic representation during optimization.

Contrastive Learning Data Augmentation +3

Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models

1 code implementation • NeurIPS 2023 • Beier Zhu, Kaihua Tang, Qianru Sun, Hanwang Zhang

In this study, we systematically examine the biases in foundation models and demonstrate the efficacy of our proposed Generalized Logit Adjustment (GLA) method.

Debiased Fine-Tuning for Vision-language Models by Prompt Regularization

no code implementations • 29 Jan 2023 • Beier Zhu, Yulei Niu, Saeil Lee, Minhoe Hur, Hanwang Zhang

We present a new paradigm for fine-tuning large-scale visionlanguage pre-trained models on downstream task, dubbed Prompt Regularization (ProReg).

Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning

no code implementations • 10 Dec 2022 • Chen Chen, Yuchen Hu, Qiang Zhang, Heqing Zou, Beier Zhu, Eng Siong Chng

Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition.

Audio-Visual Speech Recognition reinforcement-learning +3

Prompt-aligned Gradient for Prompt Tuning

1 code implementation • ICCV 2023 • Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, Hanwang Zhang

Thanks to the large pre-trained vision-language models (VLMs) like CLIP, we can craft a zero-shot classifier by "prompt", e. g., the confidence score of an image being "[CLASS]" can be obtained by using the VLM provided similarity measure between the image and the prompt sentence "a photo of a [CLASS]".

Domain Adaptation Few-Shot Learning +2

Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification

1 code implementation • 29 Dec 2021 • Beier Zhu, Yulei Niu, Xian-Sheng Hua, Hanwang Zhang

We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of tail over head, as the test distribution is assumed to be balanced; however, when the test is as imbalanced as the long-tailed training data -- let the test respect Zipf's law of nature -- the tail bias is no longer beneficial overall because it hurts the head majorities.

Classification

Fast and Accurate: Structure Coherence Component for Face Alignment

no code implementations • 21 Jun 2020 • Beier Zhu, Chunze Lin, Quan Wang, Renjie Liao, Chen Qian

In this paper, we propose a fast and accurate coordinate regression method for face alignment.

Face Alignment regression