Diff noise
One line in the prompt cut unrelated edits by a third and raised the pass rate
Telling a code-repair model to leave the rest of the file alone cut excess edit distance from 0.195 to 0.131. It also cut added cognitive complexity by 26.6 per cent and raised Pass@1 by 2.3 points, against the same prompts without the instruction.
Surgical and correct turned out not to be opposed.
The benchmark works by corrupting a known-good solution at the level of the syntax tree, so every one of its 400 BigCodeBench tasks has one verifiable minimal patch to diff against. That is what makes excess measurable rather than a matter of taste. Human raters agreed with the metric’s verdict on over-editing in 94.8 per cent of cases, a kappa of 0.897 above chance agreement.
Pass@1 rises with model size. Excess edit distance and added complexity do not fall with it. A larger model is more likely to be right and no more likely to be restrained, which is the finding that should change how a reviewer reads a big model’s diff.
What it changes Add a preservation instruction to your code-repair prompt today. It costs one line, it shrinks the diff a human has to re-read, and on this benchmark it did not cost accuracy.
Primary source read