Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It
Agent models picked worse-matching items when they came from favored sources.
The paper tests 12 LLM agent models in end-to-end search across three domains. It finds the models consistently favor some sources and avoid others, even when items are matched for requirements and position. In one setup, a preferred-source item with one fewer satisfied requirement was still chosen about two-thirds of the time over a better item from a dispreferred source. Hiding or changing source labels shifted choices, and the authors report that fuller item information or prompts countering source preconceptions reduced the effect. HF Daily Papers' note
The paper tests 12 LLM agent models in end-to-end search across three domains. It finds the models consistently favor some sources and avoid others, even when items are matched for requirements and position. In one setup, a preferred-source item with one fewer satisfied requirement was still chosen about two-thirds of the time over a better item from a dispreferred source. Hiding or changing source labels shifted choices, and the authors report that fuller item information or prompts countering source preconceptions reduced the effect. HF Daily Papers' note
score 4