Discovering hidden visual concepts beyond linguistic input in Infant learning

Abstract

Infants develop complex visual understanding rapidly, even preceding of the acquisition of linguistic inputs. As computer vision seeks to replicate the hu- man vision system, understanding infant visual development may o”er valu- able insights. In this study, we present an interdisciplinary study exploring this question: can a computational model that imitates the infant learning process develop broader visual concepts that extend beyond the vocabulary it has heard, similar to how infants naturally learn? To investigate this, we analyze representation from a recently published model in Science by Vong et al.[1], which is trained on longitudinal, egocentric images of a single child paired with transcribed parental speech. We introduce a training-free framework that can discover and utilize visual concept neurons hidden in the model’s internal representations. Our findings show that these neurons can classify objects beyond its original vocabulary. Furthermore, we compare the visual representations in infant-like models with those in modern computer vision models, such as CLIP or ImageNet pre-trained model, highlighting key similarities and di”erences. Ultimately, our work bridges cognitive science and computer vision by analyzing the internal representations of a computational model trained sorely on an infant’s visual and linguistic inputs.