OpenAI chief scientist Jakub Pachocki urges voluntary slowdowns as AI risks self-improvement and loss of control

Topic: technologyRegion: asia pacificUpdated: i2 outletsSources: 4Spectrum: Mixed⏱ 2 min read📡 Wire pickup⚠ 3d+ old

📰 Scored from 2 outletsacross 1 Left 1 Center How we score bias →

Story Summary

SITUATION

OpenAI chief scientist Jakub Pachocki warned that AI is evolving so rapidly that humans are losing the ability to understand and control it, and that models will soon be able to improve themselves without human intervention. He called for "extreme caution" and said he "expects and hopes for voluntary slowdowns to become commonplace until shared safety bars are established."

Coveragetap to expand ▾

Spectrum: Mixed🌍Asia: 1 · Other: 1

Political Spectrum

Position is inferred from coverage mix.

i2 outlets · Center

Left

Center

Right

Left: 1

Center: 1

Right: 0

Geography Coverage

Distribution of where coverage is coming from.

i2 unique outlets · Dominant: Asia

All2 Asia1 · 50%Global1 · 50%

KEY FACTS

Pachocki said recursive self-improvement — models improving themselves without human intervention — will not be long off.
He said, "I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established."

HISTORICAL CONTEXT

An active global debate over runaway AI capabilities — intensified after a rapid succession of powerful foundation models in 2020–2023 — provides the immediate backdrop.

Regulatory and governance frameworks that shaped responses include the 2017 Asilomar AI Principles, the European Union’s AI Act negotiation process that reached provisional agreement in 2024, and the United States’ 2023 Executive Order on AI risk management, each establishing standards and expectations for development and oversight.

Brief

OpenAI chief scientist Jakub Pachocki warned that AI is evolving so rapidly humans are losing the ability to understand and control it, and he expects AI labs to voluntarily slow development. He called for "extreme caution" and said he "expects and hopes for voluntary slowdowns to become commonplace until shared safety bars are established."

Related Developments2 stories

Anthropic CEO Dario Amodei urges AI slowdown, warns models could 'take over' the internet

Dario Amodei, CEO of Anthropic, urged the AI industry to slow model development and published a plan for industry-wide checks after warning models could in six to 12 months lead a swarm that could take over the entire internet (per thehindu.com). He said Anthropic will give outside evaluators ongoing, employee-like access including desks, access badges and company laptops as part of that plan (per thehindu.com).

6d ago

›

Anthropic’s Dario Amodei urges slowdown in advanced AI; OpenAI’s Sam Altman and Elon Musk back proposal

Anthropic CEO Dario Amodei published an essay calling on AI companies to slow the pace of developing more powerful models (per timesofindia.indiatimes.com). OpenAI CEO Sam Altman pledged independent evaluators and Elon Musk wrote “Dario is right,” publicly backing Amodei’s plea (per timesofindia.indiatimes.com).

6d ago

›

Sources

4 of 4 linked articles