Slop, productivity, and why the AI-fueled world is going nowhere mighty fast
Just saw a graph at the FT from John Burn-Murdoch that really distills something I have been trying to articulate. AI generates a lot of output (which fits many people’s informal notion of productivity), but it hasn’t yielded much in the way of RoI for many companies (per studies from MIT, McKinsey, Bain, and many others), and hasn’t materially changed GDP.
The particular graph from the FT happens to be about mobile apps (h/t Jen Zhu). But you could make similar graphs in many domains: loads of nominal productivity, but not much real world impact.
You could, for example, say something similar about books. GenAI has led to a lot more books, but it’s not clear that they are better books or (sorry I don’t have the data immediately available) more books sold. Here are numbers of books available over time, from The Washington Post:
Book sales, on the other hand, have declined slightly over the same period. And I don’t hear any argument that books are getting better.
The Post article that I drew the above from had similar graphs for the rise in music tracks uploaded, self-represented lawsuits filed, scientific papers submitted, and web content generated.
Again there is no reason to think that any of it has added to the GDP or quality of science or music. Most of it is slop (and in some cases work slop):
The graphs from FT and The Washington Post show the same thing over and over; we are being flooded with AI-generated content, from apps to music to scientific papers, but that doesn’t make any of it any good. Most of it (in fairness, not all) isn’t.
And then you have slop inundating Wikipedia, libraries and so on.
It is very likely that the same thing is happening in math. A group of mathematicians, who are fearful about many aspects of the impact of AI on math research, have written an open letter called the “Leiden Declaration”, reported by the NYT, expressing (among other concerns) their fear that
Current automated techniques can produce plausible but unreliable (or even incorrect) arguments which are difficult to distinguish from correct mathematical proofs. This applies not only to informal arguments, but also to formalizations, where the difficulty lies in the translation between computer-encoded and human presentations of concepts. These fast-moving developments put our present system of review under increasing pressure, jeopardizing our ability to implement traditional standards for the correctness, transparency, and independent verifiability of proof.
In other words, math slop.
Not long ago, when I was thinking about the official productivity measures and whether they would change with AI, an investor friend wrote to remind me how narrow the technical definition of productivity with respect to GDP is, noting that “In the technical world of national accounting, paying people to dig and refill holes adds to GDP.”
Using GenAI runs up tokens costs, and floods the world with content. But all too often that content — created by a giant but unreliable word prediction machine that is more regurgitative than innovative — is of little lasting value.
Maybe the biggest exception is coding, but even there, it’s not yet clear how many of the systems that have been created with the help of agentic coding will endure.
§
Moreover, especially in agentic coding, there are massive losses, up and down the food chain. The providers (Open AI, Anthropic, Cursor, etc) lose tons of money, and are scrambling to raise prices, Customers are balking at the new usage models.(In a new analysis, Gerben Wierda has argued that Anthropic and OpenAI “may actually pay $1000 for every $100 you pay them”. Data center intermediaries like CoreWeave are also losing money. It’s hard even to say how the chip companies are doing, once you factor out all the circular financing and shrinking cash flow. The one possible major productivity story (coding) also happens to be the one burning the most cash.
It could well be that once the providers (Open AI, Anthropic, Cursor, etc) attempt to charge enough money to cover their losses, the cost of AI could become more expensive than the humans it is replacing.
Talk about digging holes without creating real value.
PS. See also Paul Krugman and Paul Kedrosky for new pieces reaching somewhat similar conclusions, from different angles.
PPS Apropos tweet, tongue presumably partly in cheek, from yesterday:
As the great basketball coach John Wooden used to say, 'Never mistake activity for achievement.'
Yep, 100%
AI allows you to easily scale the volume of output. However, it doesn't allow you to easily scale things of value.
Businesses measuring productivity gains by LOC (lines of code) are likely reading the wrong signal which is going to eventually be quite painful.