Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting

ECCV 2020  ·  Shengcai Liao, Ling Shao ·

For person re-identification, existing deep networks often focus on representation learning. However, without transfer learning, the learned model is fixed as is, which is not adaptable for handling various unseen scenarios. In this paper, beyond representation learning, we consider how to formulate person image matching directly in deep feature maps. We treat image matching as finding local correspondences in feature maps, and construct query-adaptive convolution kernels on the fly to achieve local matching. In this way, the matching process and results are interpretable, and this explicit matching is more generalizable than representation features to unseen scenarios, such as unknown misalignments, pose or viewpoint changes. To facilitate end-to-end training of this architecture, we further build a class memory module to cache feature maps of the most recent samples of each class, so as to compute image matching losses for metric learning. Through direct cross-dataset evaluation, the proposed Query-Adaptive Convolution (QAConv) method gains large improvements over popular learning methods (about 10%+ mAP), and achieves comparable results to many transfer learning methods. Besides, a model-free temporal cooccurrence based score weighting method called TLift is proposed, which improves the performance to a further extent, achieving state-of-the-art results in cross-dataset person re-identification. Code is available at https://github.com/ShengcaiLiao/QAConv.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Results from the Paper


Ranked #3 on Generalizable Person Re-identification on Market-1501 (MSMT17-All->mAP metric, using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Generalizable Person Re-identification CUHK03-NP (detected) QAConv MSMT17-All->mAP 22.6 # 3
MSMT17-All->Rank-1 25.3 # 3
Market-1501->mAP 8.6 # 3
Market-1501->Rank-1 9.9 # 3
Generalizable Person Re-identification Market-1501 QAConv MSMT17-All->mAP 43.1 # 3
MSMT17-All->Rank-1 72.6 # 3
Generalizable Person Re-identification MSMT17 QAConv Market-1501->Rank1 22.6 # 3
Market-1501->mAP 7.0 # 3

Methods