Fig. 5

A depiction of the forward pass for a positive pair in a standard noncontrastive pretext task. An image is subject to stochastic data transformations twice, producing distorted views \(\textbf{x}_a\) and \(\textbf{x}_b\), which are passed through the feature extractor \(f_\theta\) to yield feature representations \(\textbf{h}_a\) and \(\textbf{h}_b\). The projector \(g_\phi\) transforms \(\textbf{h}_a\) and \(\textbf{h}_b\) into embeddings \(\textbf{z}_a\) and \(\textbf{z}_b\) respectively. Typically, the objective \(\mathcal {L}\) is optimized to maximize the similarity of \(\textbf{z}_a\) and \(\textbf{z}_b\)