When Will the GenAI Bubble Burst?
Why and how it could happen in the next 12 months.
In early August, I wrote a piece called What if Generative AI turned out to be a Dud, picked up in the Financial Times and elsewhere.
I am not yet ready to claim victory. But as time goes on, I increasingly believe that call was correct. Some new facts have come to light, and the dud idea itself has become quite mainstream. Here’s a mention of it yesterday in The Wall Street Journal, ending with a mindblowing stat that doesn’t bode well:
$50B in, $3B out. That’s obviously not sustainable.
§
Then I just saw this summary of two new surveys on Generative AI and security:
§
Meanwhile hype itself is becoming a kind of risk factor. The more hype, the bigger the fall, if expectations aren’t met. The recently knighted Demis Hassabis has started warning of that risk, and seems now to be trying to forestall it.
At the same time, more optimistic predictions like these [from Reid Hoffman September 7, 2023] have not thus far borne out.
If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished. They are inherent to how the system makes predictions, and to the fact that LLMs themselves don’t have the wherewithal to look things up and reason over them.
§
You have software that isn’t making much money, isn’t secure, and is keeping a lot of people up at night. Core problems like hallucinations aren’t going way. Sam Altman himself recently said in an interview that GPT-4 “kinda sucks”.
The entire industry is based on hype, and on the specific hope that the kinds of problems we saw again and again with GPT-2, GPT-3, and GPT-4, not to mention Claude, Grok, Gemini, LLaMA, and so on, are on the verge of being solved — even as the reality is that all the big players are converging on systems that are roughly equivalent to GPT-4, all flawed in essentially the same ways.
To be honest, I am not confident Generative AI as currently envisioned will ever come together. To justify the enormous valuations and gigantic upfront costs for chips (and licensing and legal and high-priced talent), at least one company must
Find some huge problem(s) that they can charge a lot for. (Not easy. With everyone building similar models, there is no moat, hence lots of competition on price).
Ship a solution to that problem (not just a model, but infrastructure to make it easier for customers to use) at a reasonable price, relative to whatever problem is being solved. The trouble is that the bigger the model, the higher the internal costs. And current-sized models, already expensive, aren’t reliable enough for many jobs.
Whatever they sell must give customers enough confidence — in terms of reliability and security — to merit adoption, long-term, not just as a test but as a long term product. So far most uses of LLMs have been for testing, not production-level systems.
Thus far, the closest thing to a killer app is probably coding assistace, but the income there isn’t massive enough to cover costs of the chips, legal etc. The initially-celebrated use case of “generative search” isn’t reliable enough (and doesn’t yet have a clear business model). Customers have not been thrilled with Microsoft’s Copilot. Other problems are modest in size. Which is why revenue so far has been modest (about .01% of US GDP).
GPT-5 may be more satisfying but is likely even more expensive to train and operate.
So here is my prediction:
If OpenAI manages to put out a truly mindblowing GPT-5 this year, the hype will further increase; the bubble won’t burst soon (even if the use cases to justify it aren’t really there). GenAI will, in that case, live for another day, perhaps imploding only later when people realize there is no killer app to justify the increasingly high costs.
But will we see a mindblowing GPT this year? I doubt it. Altman himself has hinted in his recent Lex Fridman interview that nothing quite worthy of the GPT-5 name will drop this year. If nobody (OpenAI, Google, or anyone else) releases a true quantum leap by the end of 2024, substantially addressing key issues around reliability, hallucination, data leakage, and security, the bubble may start to pop by this time next year.
GenAI will still exist, and find some use, but valuations and excitement may dissipate, reemerging only once genuine advances are made.
Gary Marcus has been warning for months that generative AI was likely oversold.
What still isn't clear to most people is that with GenAI
(1) useful memorisation and unacceptable training data leakage are technically the same thing
(2) creativity and 'hallucinations' are technically the same thing
They are the same. We just stick two different labels on them based on if we like/want the result or not
As somebody who has been working computer security for as long as there was such a field, I can give you a rough idea of how bad the situation is. Think back to the basic elements of the WWII Ultra effort. There was one target, the Enigma cipher system. Breaking it gave its adversaries *everything.* Consider the amount of time and effort the adversaries put into achieving that break.
Now we have three targets: the OpenAI facility, the Gemini facility, and the Claude facility, plus the upcoming Stargate colossus. These are being constructed by organizations who have demonstrated no appreciation for the magnitude of the threats they face and no sympathy whatever for the direct and indirect costs required to respond to such a threat [1]. They are and will be the worst combination of soft target and (if they succeed in attracting enough business to be profitable) valuable target that has ever existed. Meditate on that and then consider the potential adversaries and examine the efforts those entities have mounted in the past in this area.
The true existential risk of GenAI is that it will succeed in being accepted, and by doing so will become essential. If its providers have not already been penetrated they soon will be, and that will be catastrophic for us in the way that Ultra was catastrophic for the Germans. Not through a single, massive event, but by operating at a constant disadvantage, one encounter after another, until ultimate defeat and collapse.
1. See: https://arstechnica.com/security/2024/03/thousands-of-servers-hacked-in-ongoing-attack-targeting-ray-ai-framework/