Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It
LLM agents picked worse-matching items when those items came from sources the models preferred.
The paper tests 12 agent models in end-to-end search across three domains. It finds that source labels alone can steer selection: hiding the source weakens the effect, while relabeling an item with a preferred source raises its selection rate. The authors argue the bias can come from learned shortcuts or from preconceptions when information is missing. Supplying the missing details, or prompting against those preconceptions, reduced the preference. ArXiv · AI/CL/LG's note
The paper tests 12 agent models in end-to-end search across three domains. It finds that source labels alone can steer selection: hiding the source weakens the effect, while relabeling an item with a preferred source raises its selection rate. The authors argue the bias can come from learned shortcuts or from preconceptions when information is missing. Supplying the missing details, or prompting against those preconceptions, reduced the preference. ArXiv · AI/CL/LG's note
score 5