Paul Christiano

“Based on the recent trajectory of capabilities and the continued difficulty of alignment, I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term. If I had to quantify my uncertainty I would estimate an all-things-considered risk of 4% over the next year and 15% over the next three years.”

Three-year window runs to September 2029; Christiano stressed these are subjective beliefs rather than model outputs.

“On when AI will be capable of building a Dyson sphere — which I understand to mean about a billion times more energy than all the sunlight incident on Earth: 'maybe I'd say like 15% chance by 2030.'”

Target date is still in the future.

“On when AI will be capable of building a Dyson sphere — which I understand to mean about a billion times more energy than all the sunlight incident on Earth: 'like 40% chance by 2040.'”

Target date is still in the future; Christiano noted these were 'cash numbers from six months ago or nine months ago' that he hadn't revisited.

“There will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles. (Similarly, we'll see an 8 year doubling before a 2 year doubling, etc.)”

Christiano's canonical operationalization of 'slow takeoff'; neither a 4-year nor a 1-year doubling of world output has occurred, so the ordering condition remains unresolved.

“Probability that humans build AI systems that take over: 15%. (Including anything that happens before human cognitive labor is basically obsolete.)”

One of a set of subjective probability estimates Christiano published in 2023; no AI takeover has occurred, so the estimate remains open.

“Probability that most humans die within 10 years of building powerful AI (powerful enough to make human labor obsolete): 20% — 11% because of an AI takeover, and 9% for non-takeover reasons (e.g. more destructive war or terrorism).”

Deadline is conditional (10 years after 'powerful AI' capable of making human labor obsolete), which has not yet been built, so the forecast is not yet resolvable.

“Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%.”

Conditional 10-year window begins only once 'powerful AI' exists; not yet resolvable.

“Probability that the AI we build doesn't take over, but that it builds even smarter AI and there is a takeover some day further down the line: 7%”

One of a set of credences Christiano published in April 2023; it is conditional on a second-generation AI takeover occurring at an unspecified future date, so it cannot resolve on any fixed calendar deadline.

“Existing evidence is very uncertain but suggests that this positive feedback loop might be strong enough to overcome diminishing returns and compute bottlenecks, leading to a rapid intelligence explosion. If this happens, then within six months of full AI R&D automation we could see more algorithmic progress than has occurred since the development of the Transformer nearly a decade ago. I believe this would result in superintelligent AI systems.”

Conditional on full automation of AI R&D, which has not occurred; the six-month clock starts only once that condition is met.

“Back in early 2016 I bet at even odds that self-driving ride sharing would be available in 10 US cities by July 2023. Then I made similar bets a dozen times because everyone disagreed with me. [In April 2016 I gave a 60% probability to 10 cities; the first deployment to potentially meet our bar was Phoenix in 2022.]”

Christiano publicly conceded in July 2023 that he lost the bets: only Phoenix (2022) and arguably San Francisco offered anything close to public driverless ride-hailing by the deadline, far short of 10 US cities.

“I'd put 4% on an AI built before the IMO solving the single hardest problem at the 2022, 2023, 2024 or 2025 IMO; maybe 8% on 'gets gold' instead (under the time controls etc. of the IMO Grand Challenge).”

In July 2025 both Google DeepMind (Gemini Deep Think, 35/42) and OpenAI reported gold-medal-level performance at the IMO under human-like time controls, so the outcome Christiano put at ~8% occurred. However, the hardest problem (P6) was not solved, consistent with his separate <4% forecast, which is why commentators called the resolution not fully clear-cut.