Methodology
The Index exists to answer a simple question: when prominent people make confident public claims about the future of AI, how often are they right? To answer it fairly, we follow a few rules.
What qualifies as a prediction
- Public and on the record. The claim was made in a publication, talk, interview, or official communication we can cite.
- Falsifiable. It names a concrete outcome and an explicit or clearly implied timeframe. “AI will change everything” doesn’t qualify; “AI will outperform radiologists within five years” does.
- Consequential. Made by someone whose views move markets, policy, or public understanding — researchers, executives, economists, public intellectuals.
How outcomes are scored
The predicted outcome happened, substantially as described, by the stated deadline.
The direction was right but the magnitude or timing was materially off — or the claim came true only in a narrower form than stated.
The deadline passed and the predicted outcome clearly did not happen.
The deadline hasn't arrived yet. We track it and resolve it when the clock runs out.
The hit rate
The headline number on the front page is the share of resolved predictions that came true, counting a partially-correct call as half a point. Pending predictions are excluded until their deadline passes — being early isn’t the same as being wrong, and we don’t resolve a claim before its clock runs out.
Caveats, honestly stated
- Selection bias is real. Memorable predictions — especially spectacular misses — are more likely to be recorded. The Index is a curated record, not a random sample of all forecasts ever made.
- Resolution requires judgment. Some claims are fuzzy at the edges. Each entry includes a resolution note explaining the call, and we cite sources so you can disagree with us on the merits.
- Probabilistic claims are tracked, not scored. A “10–20% chance” claim can’t be marked right or wrong by a single outcome; we record it for the historical record.