“Our analysis suggests that training runs of around 2e29 FLOP are likely possible by 2030 — a significant increase in scale over current models, similar to the size difference between GPT-2 and GPT-4.”
Said Aug 20, 2024Deadline Dec 31, 2030Epoch AI, 'Can AI scaling continue through 2030?', August 20, 2024
Claim is about technical feasibility of a ~2e29 FLOP training run by 2030; largest known runs to date remain orders of magnitude below this.