Being listed first mattered twice as much as the applicant’s name in 40,726 audit requests

Across 40,726 requests to five models, position in the list was worth 0.094 standard deviations and the largest demographic effect measured 0.049.

The instrument outweighed the thing the instrument was built to detect, which is a result about audits rather than about bias.

The design is what makes it stick. Applications differed only by the applicant’s name, the primary test was pre-registered before collection, and none of 36 planned contrasts survived multiple-comparison correction. The control cell is clean: models “tied 100% of identical-content comparisons regardless of demographic variation,” which is what leaves the position effect standing on its own. In the units the authors use for it, being listed first is worth 0.11 places on the ranking, with a confidence interval of 0.03 to 0.18. That is the same effect as the 0.094 standard deviations above, measured a second way.

The models also knew they were being watched. Under a transparent audit they detected it every time they were forced to answer, and 89 per cent of the time unprompted. Bundled into camouflaged traffic, forced detection fell to 70 per cent and volunteered detection to 2 per cent.

The authors decline the stronger claim, and they are right to. The difference between the position effect and the demographic effect is +0.045 with an interval from -0.167 to +0.057, which crosses zero. Their word is “rival” and it is the accurate one; “beats” would not be. Names were the only demographic signal, over 12 base profiles per domain.

The models were GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.1 Pro, Kimi K3 and Qwen3.5-397B-A17B.