Promises are cheap
Nope. This is not an essay about Elon Musk. It’s also not about how in 2019 he promised a million robotaxis in 2020.
Or even how he eventually walked that back to 2026 (which also won’t happen).
Nope, it’s about Microsoft’s AI CEO who (following Elon’s playbook) just made big promises to the FT:
who happens to also be this guy:
Now, as far as I know, LLM hallucinations are still a serious problem. And as far as I know, a lot of lawyers who use LLMs are having problems with hallucinations. (When I wrote about them less than a year ago there were 112 documented cases with lawyers who got called out in Damien Charlotin’s database; now there are 914. That’s 8x growth in less than 12 months!)
Do you want to use an accountant who hallucinates? Or who has the kind of pervasive troubles with reasoning that researchers at Caltech and Stanford just documented? Back on planet earth, according to December’s Remote Labor Index, only 2.5% of human “online tasks” (physical tasks were excluded) could be done by AI.
A decade ago, Geoff Hinton, then working for Google, said “We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists.” (In fact, the number of radiologists keeps growing, and in some communities there is a shortage.)
Promises are cheap. And all the big tech CEOs have come to realize that making them costs them nothing. (Some CTOs have caught too; Suleyman’s colleague Kevin Scott hinted that GPT-5 would blow away PhDs, and be vastly better than GPT-4; in reality the improvements were far more modest.)
Tesla trades at nearly 400 times earnings not because they have ever made all that much money, but because Elon is a master of hype. Others like Altman and Amodei and Suleyman have taken note, and play the same game.
It would be nice if places like FT would refuse to play along, providing more context (about conflicts of interest and past prediction track records). It would also help if they would more regularly seek independent outside opinions. Instead, too often media companies platform predictions from people who stand to gain immensely from their narratives, with too little skepticism.
The public has not been well served by this practice. If the whole thing collapses, as many people fear, the careless repetition of hype by editors who love the “everything is about to change” narrative (always good for clicks) may well turn out to have been an important contributing factor.
Unfortunately Nobody ever got attention by predicting stuff will roughly be the same 😂
I actually have some hands on experience with legal AI over the last 6 months. The newer models, combined with pretty extensive guidelines as context engineering, do a pretty good job of legal drafting, at least in the contracts area where I was working. The problem is that fine tuning them to get correct drafts first time is not simple, and the legal profession tends to be very conservative, for all sorts of reasons. That experience, and others in other domains where people are trying to apply AI, has convinced me that commercial uptake of AI in office work will be much slower and more gradual that the hype merchants are suggesting.
I also am a software developer with considerable experience using Claude. That experience has taught me that coding is the best current use case for AI, and there is a revolution already under way. The latest tools like Claude Code produce flawless code in seconds. But the trick is to know exactly how to craft a spec for an LLM. That takes experience (which I have), and the latest research is now suggesting that it is the experienced developers, not the vibe coders, who are getting the most benefit from the tools. That's because we know that coding is not the whole job, or even the largest part of the job. The real job also includes requirements gathering, specification, bench checking, testing and documentation. So in coding, the benefits are real, they are here now. But they are not a replacement for traditional IT skills, They are a productivity tool for people who already have the skills.