The veto gate
A 4B draft model reading a coding agent’s output cut execution errors and token spend, and task success slipped
Konstantin Grotov and Valentin Malykh inverted speculative decoding: a small draft model scores a coding agent’s finished trajectory instead of proposing tokens for it.
The signal is real, and the gate that uses it is honest about what it does not buy.
A Qwen3-4B draft, distilled on OpenHands trajectories from SWE-rebench, reads the black-box agent’s output text in one teacher-forced forward pass, about 100 milliseconds on one A100 against the far larger latency of the agent’s own step, and returns per-token cross-likelihoods. The method splits the trajectory into a high-entropy reasoning span and a low-entropy action span pinned down by tool-call syntax, extracts features from each separately, and fits a linear calibrator to whether the next action will execute without an exception. Averaging over the whole trajectory, as the prior HTC method does, dissolves the signal.
As a pre-execution veto gate on Qwen3-Coder-480B and Claude 3.5 Sonnet, it moved the two numbers a team is accountable for. Per-call execution error on SWE-Bench Verified fell to 15 per cent, down from 21, and on DA-Code to 6 per cent, down from 14, the second without retraining the calibrator; average tokens per task fell 14 to 19 per cent. Resolved rate on SWE-Bench Verified slipped to 44 per cent, down from 49, and DA-Code to 63, down from 64, because a replan is not guaranteed to recover and some vetoes block actions that would have worked.
The authors report single-run estimates and one objective, exception-free execution, and say the draft’s cost is objective-dependent.
What it changes Worth a week’s experiment if you pay for retries at volume: the recipe is a few thousand labelled trajectories, a distilled 4B draft and a linear calibrator, and the return is fewer wasted execute-fail-retry loops, not a higher benchmark score.
Primary source read