Groupthink versus Tulips. What will historians think of 2024?

Some thoughts occasioned by Meta’s new model and a bad week in the stock market

One of the bigger AI items of the week was that Meta released the first draft of an open source model, LLAMA-3. Like so many recent models, the new model approached GPT-4 but didn’t exceed it, consistent (but not proof of) my ongoing conjecture of diminishing returns.

Strikingly, Meta is giving it away free, and the more deeply-trained version (still to come/still being trained) may well be in the general vicinity of GPT-4.

That means, of course, that OpenAI’s business model is threatened; same for Anthropic’s, Mistral’s, and so on. Whatever tiny moat there was between the companies will further diminish. LLMs are well on their way to becoming a commodity; price wars will be intense.

Meanwhile, and perhaps not coincidentally, the bubble deflation that I have been warning since August of has finally perhaps begun:

A new Washington Post column makes a case similar to the one I have been making:

So 2024 could be remembered as the high point in a kind of tulip-mania.

§

But my biggest thought on the situation (perhaps stoked a little bit by Filip Piekniewski’s related thought yesterday) is about groupthink: all the major AI companies are spending billions producing almost exactly the same results using almost exactly the same data using almost exactly the same technology — all flawed in almost exactly the same ways.

Historians are going to be scratching their heads.

Gary Marcus is shaking his head, but cautiously optimistic that an increase of realism might lead to a renaissance of new approaches to AI.

.