Open-vocabulary object detection via debiased curriculum self-training

Abstract

Open-vocabulary object detection aims to train a detector capable of recognizing various novel classes. Most existing studies exploit image-level weak supervision to generate pseudo object boxes for novel class training. However, the generated pseudo boxes are often noisy and biased towards base classes, leading to sub-optimal open-vocabulary detectors. We propose DCS, a novel Debiased Curriculum Self-Training technique that generates pseudo object boxes progressively and adaptively for training accurate open-vocabulary detectors. DCS consists of two complementary designs, namely, progressive pseudo-label filtering (PPF) and adaptive pseudo-label selection (APS). Specifically, PPF discards confident but mismatched detection progressively at the early training stage when the trained detector is biased towards the base classes, APS instead fuses class-aware and class-agnostic pseudo labels by prioritizing class-aware pseudo labels at the late training stage when the detector can better recognize novel classes. Without bells and whistles, DCS achieves superior detection performance over two open-vocabulary detection benchmarks.