AGI by 2027?

Fun with charts

Apparently OpenAI’s internal roadmap alleged that AGI would be achieved by 2027, according to the suddenly ubiquitous Leopold Aschenbrenner (recently fired by OpenAI, previous employed by SBF), who yesterday published the math to allegedly justify it:

I was going to write a Substack about how naive and problematic the graph was, but I am afraid somebody else has beaten me to it:

§

Well, ok I will add a few words:

• GPT-4 is not actually equivalent to a smart high schooler. It can do some things better, some worse, very few reliably. Even it had access to a robot body (as is now common in some labs), I would not hire it to do most of things that high school students are often hired to do, like work as a lifeguard, work in restaurants, work in a grocery store, etc. Just because it can write average high school term papers based on massive numbers of samples doesn’t mean it’s on par with a smart high schoolers in their capacity to deal with the real world. Smart high schoolers often do original work, like competing for science fair prizes with original ideas or designing and implementing original computer systems singlehandedly. GPT-4 on its own ain’t doing any of that, as far as I know.

• The double Y-axis makes no sense, and presupposes its own conclusion. The numbers on the left Y axis (“effective compute”) and their relation to the qualitative Y axis on the right (conflating model capabilities with human capabilities in a strange linear mishmosh) are arbitrary, verging on make-believe.

• The graph assumes that all continued progress will be same — ignoring qualitatively unsolved problems (hallucinations, planning, reasoning etc), the fact that synthetic data may not be as useful as natural data, limits on then-possible energy distribution and compute, etc. Truly trillion pound baby stuff.

• GPT-4 existed in August 2022. I can’t think of a single measure in which any existing model in 2024 outpace GPT-4 by as much as GPT-4 outpaced the state of the art in 2020. We are already behind the curve.

• And honestly, every data point there is imaginary; we aren’t plotting real things here, no matter the legend might suggest.

§

All that said, Aschenbrenner does actually make and develop a very good (though not original) point in his new 165 page manuscript: we are (as Eliezer Yudkowsky has rightly argued for years) woefully underprepared for AGI whenever it comes. If you read his manuscript, please read it for his concerns about our underpreparedness, not for his sensationalist timelines.

The thing is, we should be worried, no matter how much time we have.

If the fiasco that has been GenAI has been any sign, self-regulation is a farce (Aschenbrenner’s former employer OpenAI apparently can’t even keep its own house in order), and the US legislature has made almost no progress thus far in reining in Silicon Valley in. It’s imperative that we do better.

Gary Marcus wrote his new book Taming Silicon Valley in part for this very reason.