Open-world Text-specified Object Counting
Our objective is open-world object counting in images, where the target object class is specified by a text description. To this end, we propose CounTX, a class-agnostic, single-stage model using a transformer decoder counting head on top of pre-trained joint text-image representations. CounTX is able to count the number of instances of any class given only an image and a text description of the target object class, and can be trained end-to-end. In addition to this model, we make the following contributions: (i) we compare the performance of CounTX to prior work on open-world object counting, and show that our approach exceeds the state of the art on all measures on the FSC-147 benchmark for methods that use text to specify the task; (ii) we present and release FSC-147-D, an enhanced version of FSC-147 with text descriptions, so that object classes can be described with more detailed language than their simple class names. FSC-147-D and the code are available at https://www.robots.ox.ac.uk/~vgg/research/countx.
PDF Abstract
CARPK
FSC147
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Object Counting | CARPK | CounTX (uses arbitrary text input to specify object to count, used "the cars" for CARPK) | MAE | 8.13 | # 7 | |
| RMSE | 10.87 | # 7 | ||||
| Zero-Shot Counting | FSC147 | CounTX | Val MAE | 17.10 | # 1 | |
| Val RMSE | 65.61 | # 3 | ||||
| Test MAE | 15.88 | # 1 | ||||
| Test RMSE | 106.29 | # 2 | ||||
| Object Counting | FSC147 | CounTX (uses text descriptions instead of visual exemplars) | MAE(val) | 17.10 | # 9 | |
| RMSE(val) | 65.61 | # 11 | ||||
| MAE(test) | 15.88 | # 10 | ||||
| RMSE(test) | 106.29 | # 12 |
