OpenAI’s chief scientist says no lab can keep scaling at full speed responsibly for much longer

Jakub Pachocki, OpenAI’s chief scientist, published An Alien Mind on 6 September, expecting recursive self-improvement within years and asking for mandated safety bars.

Its operative sentence is a commitment, not a warning: OpenAI will unilaterally withhold further scaling as needed. The disclosure beside it shows what withholding looked like in August.

The reason he gives is specific. OpenAI’s main bet on oversight has been chain-of-thought monitoring, reading the model’s verbalised reasoning rather than supervising it, which is why o1-preview shipped with its chain of thought hidden from users. He writes that OpenAI’s evaluations indicate its ability to rely on that monitoring is progressively diminishing for the Astra class, and that he expects general AI progress to be increasingly bottlenecked by confidence in monitoring.

Based on internal results, he writes, he strongly expects the current speed to carry into recursive self-improvement within a few years. He says OpenAI focuses its research on RSI because it believes that is the only way to remain at the frontier, and asks for commitments like the Preparedness Framework and Anthropic’s Responsible Scaling Policy to become mandated safety bars enforced by third-party auditors, agencies or international bodies.

The obvious objection is that a chief scientist asking for mandated slowdowns pays nothing while his lab keeps training. The paper’s ruling is that the essay is sincere and the word is doing less than it sounds: the withholding has no stated threshold and no date, and its only instance on record is the two-week pause beside this, which moved compute rather than reducing it.