“Our analysis suggests that training runs of around 2e29 FLOP are likely possible by 2030 — a significant increase in scale over current models, similar to the size difference between GPT-2 and GPT-4.”

Claim is about technical feasibility of a ~2e29 FLOP training run by 2030; largest known runs to date remain orders of magnitude below this.