Our paper has been accepted to ICLR 2026!
In it, we tackle a long-standing problem in a popular class of self-supervised learning methods that rely on “prototypes”—reference points in representation space that help neural networks learn from unlabeled data. These methods often suffer from what we call partial prototype collapse: instead of spreading out to represent diverse patterns in the data, many prototypes end up clustering together and becoming almost identical. See the example below of how different existing methods collapse (a-d) and how our proposal fixes this issue (e).
This defeats their purpose and has pushed practitioners to use ever-larger prototype sets or hand-tuned penalties that treat the symptoms rather than the cause. By carefully analyzing how these systems learn, we show that the collapse stems from optimizing the encoder and the prototypes together: early in training, the prototypes drift into redundant configurations that conveniently reduce the loss but fail to encourage diverse representations. To fix this, we propose a simple but principled alternative: decouple the training of prototypes from the encoder altogether. We model prototypes as a Gaussian mixture that is updated with an online EM-like procedure, completely independent of the encoder’s objective. This separation prevents collapse without any extra regularization, consistently produces diverse prototypes, and, in multiple benchmarks, leads to better performance on downstream tasks.
See you at ICLR 2026!