AI Insight
Researchers developed a behavioral benchmark where over 1000 participants made judgments about whether dot pairs belonged to the same or different objects in natural scenes, then tested various AI vision models against human performance. Transformer-based models trained with the DINO self-supervised learning method showed the strongest alignment with human object perception, with this alignment correlating to how well models grouped image patches within versus between objects. The study found that matching the similarity structure (Gram matrix) of supervised models to self-supervised models improved their human-like performance, suggesting self-supervised learning captures object boundaries similarly to human vision.
Why it matters
This work provides quantitative evidence that certain AI vision systems perceive objects in ways that align with human cognition, which could improve computer vision applications requiring human-compatible scene understanding, such as autonomous vehicles, robotics, and medical imaging. The findings also suggest design principles for developing AI models with more human-like visual perception.
Understand the Science
arXiv:2603.13994v2 Announce Type: replace-cross
Abstract: Vision foundation models trained with self-supervised objectives achieve strong performance across diverse tasks and exhibit emergent object segmentation properties. However, their alignment with human object perception remains poorly understood. Here, we introduce a behavioral benchmark in which participants make same/different object judgments for dot pairs on naturalistic scenes, scaling up a classical psychophysics paradigm to over 1000 trials. We test a diverse set of vision models using a simple readout from their representations to predict subjects’ reaction times. We observe a steady improvement across model generations, with both architecture and training objective contributing to alignment, and transformer-based models trained with the DINO self-supervised objective showing the strongest performance. To investigate the source of this improvement, we propose a novel metric to quantify the object-centric component of representations by measuring patch similarity within and between objects. Across models, stronger object-centric structure predicts human segmentation behavior more accurately. We further show that matching the Gram matrix of supervised transformer models, capturing similarity structure across image patches, with that of a self-supervised model through distillation improves their alignment with human behavior, converging with the prior finding that Gram anchoring improves DINOv3’s feature quality. Together, these results demonstrate that self-supervised vision models capture object structure in a behaviorally human-like manner, and that Gram matrix structure plays a role in driving perceptual alignment.
Source: Human-like Object Grouping in Self-supervised Vision Transformers