Megadose The AI curve, compressed.

Contrastive latent representation learning with denoising score matching

· ArXiv · AI/CL/LG ·
ScoreCLR makes the repulsion cost linear in batch size, with minimal loss of linear-probe accuracy in the reported tests.

Contrastive learning typically pushes different samples apart using explicit negative examples, an approach that can require large batches and produce high-variance estimates. ScoreCLR instead learns that repulsion through a lightweight network trained alongside the encoder using denoising score matching. On CIFAR-10 and ImageNet-100, the authors report accuracy close to SimCLR under a shared training recipe. Its projector outputs also have higher effective rank and sit closer to the Gaussian prior used for regularization.

ArXiv · AI/CL/LG's note

score 5

Categories: Research