Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings
Sentence identity shows up when models compare both sentences together, not in the standalone vectors used by standard retrieval.
The paper reports weak AUCs for off-the-shelf encoders and independently encoded LM states on PAWS-X meaning-identity tests. A joint forward pass over the sentence pair reaches much higher scores, and the signal collapses when partners are shuffled. Nonlinear pair readers can recover some of the relation from independent encodings, but fixed or linear readers cannot. The author argues cosine similarity mainly tracks wording neighborhoods, while identity is a cheap computed operation. ArXiv · AI/CL/LG's note
The paper reports weak AUCs for off-the-shelf encoders and independently encoded LM states on PAWS-X meaning-identity tests. A joint forward pass over the sentence pair reaches much higher scores, and the signal collapses when partners are shuffled. Nonlinear pair readers can recover some of the relation from independent encodings, but fixed or linear readers cannot. The author argues cosine similarity mainly tracks wording neighborhoods, while identity is a cheap computed operation. ArXiv · AI/CL/LG's note
score 5