Embedding Cosine Measures Text Length Before It Measures Homogenization
Accounting for comment length erased the apparent post-2022 shift toward homogeneity in conservative Reddit discourse.
Fengming Liu re-examined a measure applied to 6.06 million Reddit comments using mean-pooled DeBERTa-v3 embeddings. Replacing comments with random ones of equal length reproduced 81% of the reported political asymmetry. On 575,000 Stack Exchange answers, the same control fully reproduced the apparent post-ChatGPT homogenization of technical sites. Corrected measures instead showed the two political camps converging between 2019 and 2022, with convergence stopping before ChatGPT.
ArXiv · AI/CL/LG's note
Fengming Liu re-examined a measure applied to 6.06 million Reddit comments using mean-pooled DeBERTa-v3 embeddings. Replacing comments with random ones of equal length reproduced 81% of the reported political asymmetry. On 575,000 Stack Exchange answers, the same control fully reproduced the apparent post-ChatGPT homogenization of technical sites. Corrected measures instead showed the two political camps converging between 2019 and 2022, with convergence stopping before ChatGPT.
ArXiv · AI/CL/LG's note
score 5