“My prediction is that in 10 years now, training data won't be important. All the large language models require scaling huge methods of data... So I think in 10 years, we'll have abilities of doing reasoning without needing to have millions of examples.”

Deadline is 2034; as of now frontier models still depend on massive pretraining corpora, though sample-efficient reasoning via RL and test-time compute is an active trend.