Will you PLEASE stop saying "coding is a practical use case"? This is the third appeal I've made on this subject. (Do you read your comments?) If you want bug ridden code with security issues which is not extensible and which no-one understands, then sure, it's a practical use case. Just like if you want nonsensical articles with invented facts, then article writing is a practical use case. But as I've pointed out already no reputable editorial is now using LLMs to write their articles. Why is that? Because it obviously doesn't work.
Let's face it the only reason you're saying "coding is a practical use case" is because you yourself don't code, and don't understand it. I can't see another reason why would assume the problems experienced in other domains somehow don't apply to coding. Newsflash: they do. And software engineering definitely doesn't need the slop any more than anyone else. So I hope this is my final appeal: please stop perpetuating this myth. If you want more information on the problems of using LLMs to code, then I can talk in great length about it - feel free to reach out. Thanks
i started coding when I was 10 years old, in languages from 6502 Assembly to C# to Python to Swift and many others. Many of my friends code professionally. I fully agree about the security risks and maintenance risks and have written about them here multiple times (eg the essay LLMs + Coding Agents = Security Nightmare). I have written about the METR study reporting noting poor results. But (a) i think you misrepresent what i have written about coding as a whole and (b) think it would be intellectually dishonest to say there is no value for coders, especially given reports on the latest iteration of Claude code.
Hi Gary - thanks for your reply. Firstly just to point out I wouldn't comment at all if I didn't already particularly respect your stance on AI in general, and more importantly the effort you've been making trying to counteract the "LLMs will produce AGI" narrative. In fact I'm pinning a lot of my hopes on you (and also Ed Zitron, whose blog is the only other place I'd comment - if he'd not switched comments off). No doubt there are plenty of other people saying the same thing (one just needs to look at the comments here, for example) but I don't think they have quite the reach or influence as yourself. (We need a hero Gary, and you're it! And I'm only half joking...)
So that's the only reason I write here - I'm otherwise really very busy (creating software!) and feel I need to avoid getting caught up in any lengthy online debates. Therefore I'm just going to write this one response and hope it covers at least some of the objections made by commenters to my original message. Apologies for not responding to everyone individually.
First I'll address your points (a) and (b):
"(a) I think you misrepresent what I have written about coding as a whole":
It is quite possible I have mistaken your position on it - but if so this was not deliberate. I was merely responding to the number of recent occasions I came across where you appear to have singled out coding as a particularly appropriate use case for LLMs. However, if I did indeed mistake your stance on this, then please could you clarify it.
"(b) [I] think it would be intellectually dishonest to say there is no value for coders, especially given reports on the latest iteration of Claude code.":
I didn't claim there is no value for coders. What I objected to is having coding singled out as a particularly appropriate use case for LLMs. There is clearly value in LLMs for any field, since for example they can quickly return a specific answer to a specific question, and this is much faster than a traditional document-based search. But this is true for everything from cake recipes to Anthropology. Similarly some commenters have pointed out that LLMs are great for getting an explanation about a complex code snippet. This is true, but once again is not restricted to coding. You could post a complex token set of any kind, including financial statements, legal documents, poetry (yes, poems can be complex and difficult to decipher!) - etc.
re. "reports on the latest iteration of Claude code" - I can't help feeling that a great many of these "reports" are more hype-driven rather than objective. (Ed Zitron agrees: "You’ve probably noticed in the last three months that everybody suddenly started talking about Claude Code, and you’d be forgiven for thinking there was some sort of massive product update rather than, as it turns out, one of the most aggressive media and astroturfing campaigns that the tech industry has ever seen." From here: https://www.wheresyoured.at/premium-the-haters-guide-to-anthropic/#how-anthropic-rugpulled-everybody-using-claude-code-deceiving-investors-the-media-and-its-users).
(Note there is actually some really quite questionable engineering that went into Claude Code itself: https://www.youtube.com/watch?v=LvW1HTSLPEk - one amusing comment on that video: "it took me 12 minutes into the video to comprehend that react has in fact been put into a terminal")
So far I've pointed out the following usages are not unique to coding:
(a) searches (I call them "tokenised searches");
(b) summarising complex token sets;
Unless I am mistaken, we are then left with (c) the question of generating code for actual use in software applications.
On this subject, firstly I'd point out the following:
"Coding" - as in Software Engineering - is actually logically isomorphic with any other engineering discipline. i.e. you could choose to code up any engineering schematic instead of e.g. drawing it on a drawing board. For example you can create a Python script for Blender and have it render a 3D plan for your house - or whatever. You could do this with engineering plans of arbitrary complexity including multi-component systems such as cars and aircraft. (I personally will not be volunteering to be the test pilot in an aircraft designed by an LLM!) We (humans) choose to draw plans first - but that's just because we find them easier to visualise. But if LLMs are particularly good for Software Engineering, they should be particularly good for these other engineering disciplines too - so instead of singling out coding, we should be able to just say "LLMs are an appropriate use case for Engineering".
That's an argument against singling out Software Engineering WRT point (c), and I'll now add just 3 points corroborating the view that Software Engineering is not an appropriate use for LLMs. I could provide many more (let me know if you want me to do that) - but this message is long already, so I'll just write down the first 3 that come to mind:
1. I mentioned that coding is actually Engineering. I think it's important to regard coding as Engineering because it makes many of the problems with using LLMs to do it instantly more obvious. Engineering is not founded on semantics, it is founded on logic. However, LLMs are good at semantics and not at logic.
LLMs have ingested a lot of code snippets from Stack Overflow, and tend to be good at rehashing those code snippets. That doesn't make them good at generalising - on a logical basis - to a much larger codebase. Humans, when they post on StackOverflow, tend to whittle the problem they are facing down to as small a representation as they can, in order for it to be appropriate for a discussion forum. Then when someone posts a solution, they take that and generalise it to their real situation. They don't post their full codebase.
As a result "successes" of LLM coding seem to be restricted to bite-sized apps which rehashing StackOverflow-type snippets can have a hope of generating. Again, based on semantics, rather than logic.
(I'm sitting in front of a large, complex codebase right now - and the thought of setting an LLM loose on it just makes me laugh!)
2. While we're on the subject of StackOverflow, prior to the advent of LLMs one could have created an app "in record time" by simply copy/pasting code snippets from StackOverflow. i.e. you don't bother to check over each line to be sure of what it does - you just go with whatever the solution is, without attempting to understand how it works or whether it's really the best solution for your particular situation. You stitch all these snippets together and voila! You have your app built in record time. Needless to say, no-one would ever recommend doing this.
The point is, there has always been a trade-off between the speed of development and quality of engineering (confidence in the code, robustness of the app etc.) I don't see LLMs as either changing this trade-off or shifting the needle (greater quality in a shorter time), because they are probabilistic and can't be relied upon to produce the best solution - or even a correct solution - every time. So you're going to have to pick your way through every single line it generates in order to have the same confidence you would have if you wrote it - and this is unlikely to save time because understanding someone else's code is always more difficult and time-consuming than writing it yourself. When I hear people say it is "making them 10x more productive" at coding, I think, "and also 10x as unsure what you've actually produced".
3. You'll also need to correct it when it does something you don't want. Now this is pretty interesting, if you think about it. Imagine you provide an LLM a prompt, and the LLM produces something but not exactly what you want. What is the advice on this? "Provide a more specific prompt!" Ok, so then we write a more specific prompt - the results are better, but it still falls short. What now? "Keep making the prompt more specific!" Ok but wait - eventually won't I be supplying the same number of tokens to the LLM as it is going to generate as the solution? Because if I'm perfectly specific about what I want, then isn't this just the same as actually writing the solution myself using a computer language? Indeed, isn't this the purpose behind computer languages in the first place? Note that in computer system development we've already addressed issues related to how specific you need to be to define a system: that's the "level" of the language; for example C is a lower level language than Python, and so would be a more appropriate choice if you need to define specifics beyond that which Python can accommodate. This "level" works all the way up to website builder applications, which are really quite high level - for when you want e.g. a payment processor integration, but you don't care about how it looks. If you say to an LLM "build me a payment processor integration" then the LLM needs to fill in the blanks and actually design that integration. Yes, you're asking the LLM to be a designer (does that seem like a good idea?) So in just the same way as a website builder, you're going to get a payment processor integration, but with no control over how it looks. That is, if it works. In the website builder case, it will definitely work. So why bother with the LLM?
Note one reason I'm taking time to try and explain my perspective on this is because I was worried about the degradation of software development long before LLMs came along. Now I think we've really hit a new peak of silliness on a scale I never imagined possible! So please understand my original message was genuinely an appeal, and not an attack. Thanks
This gets to the root of the problem as I see it. Coding isn't a thing on its own. It is the end of a long chain of reasoning and decision-making. So even if an LLM could "code" perfectly (which it can't and won't do anytime soon), there are still a huge number of things that a developer (particularly a senior one) does that aren't "coding" per se.
Think about hiring a contractor to write some software. You have to give them a spec, and unless you want the project to eat unlimited amounts of money and time, that spec will need to be quite detailed. It must explain in detail exactly what the product should do, give some amount of explanation of how it should look, what important error conditions need to be handled, what the acceptance criteria are and so on.
A full specification for a non-trivial product can run to hundreds or thousands of pages and take months to write. Even with a great spec it is common to go back-and-forth many times before final acceptance.
Current coding assistants can't do any of this, really. They help some people avoid doing what they consider "drudge work" and that's about it. The idea that developers are going to be replaced by this generation of AI is hogwash.
Which isn't to say the absurd hype-level won't lead a lot of companies down the garden path to ruin. It will.
He said "coding is a practical use case", not that it is the only practical use case, so arguments that it's not alone are *non sequitur*. And if one looks at who is making serious use of LLMs, who is paying Anthropic et. al. for tokens, coding predominates. So a demand that he stop saying that "coding is a practical use case" is not well founded.
> As a result "successes" of LLM coding seem to be restricted to bite-sized apps which rehashing StackOverflow-type snippets can have a hope of generating.
This is not true.
> prior to the advent of LLMs one could have created an app "in record time" by simply copy/pasting code snippets from StackOverflow.
Nor this. An example is the translation of the Ladybird browsers from C++ to Rust in weeks rather than months. Another example is https://github.com/ecto/loon
You should consider combing through Hacker News to see how people are actually making successful use of LLMs (there are also many forms of misuse, abuse, and failure). It would be time consuming and I wouldn't blame you if you don't, but without doing so your statements are not grounded in fact.
Note: I am not endorsing the use of LLMs in coding, for a number of reasons including the economic and environmental impact of this technology. I am only addressing the facts about the use of that technology, and one of the facts is that coding is "a practical use case", for a properly understood notion of "practical".
P.S. Speaking of "practical", this just landed in my inbox:
1. "coding is a practical use case" I was referring to the fact Gary has singled out coding as an example of an appropriate use on multiple occasions, and not the literal meaning of this one particular reference.
2. "And if one looks at who is making serious use of LLMs, who is paying Anthropic et. al. for tokens, coding predominates." - The fact that lots of people are trying to use LLMs to code does not make it a good idea, or mean they are successfully building high quality, robust software.
3. "This is not true". You say it's not true; I say it is true. All you need to do is download Claude Code and try and build anything with it, and it is immediately apparent this is the case.
4. "Nor this." Actually this is very much the case. We software developers very often pull chunks of code from various locations - not just stackoverflow. Very often they are chunks of code we wrote ourselves, that we then adapt to the new system we are inserting it into. This is great, because we don't need to make an effort to understand the code we're inserting - we already understand it, because we wrote it. If we lift code from somewhere like stackoverflow - or take it from some open source module on github etc, then it's very important we make an effort to understand how it works, because it might not be completely appropriate to the specific location within the system we're inserting it into, and may therefore work as expected in some circumstances but not in others. One can indeed not bother to try to understand chunks of code that one inserts, and pay the price of not being sure that code is going to work correctly as a result. As I mentioned, the point I was making is that there's always been a tradeoff between speed of construction and confidence in the robustness of the solution (I am sure you don't deny this is the case). Furthermore I think that inserting code chunks without checking you understand them is actually very similar to coding with an LLM - the only difference being the LLM generated the code chunk instead of you finding it online somewhere.
5. "You should consider combing through Hacker News to see how people are actually making successful use of LLMs" - the problem with this is there are really a lot of hype-driven stories out there that are basically made up. I've caught some that are obvious - e.g. see my comment on this post: https://substack.com/home/post/p-185469925 - which then makes me quite sceptical of many of the others. I'm not really sure why this kind of fabrication has become so prevalent - I find it very strange - but there's certainly a lot of it going on. At the end of the day I'm going to trust my own experiences actually trying to use these tools, and not stories about them that I can't verify.
Sigh. I've martyred myself in this cause. I got fired from a job in 2009 because I refused to incorporate 3rd party code from an unvetted origin (management found it through a web search). Well, I would have done it if they'd given me days to study the code and days to test the integrated result. But they wanted it done in an afternoon. Fortunately, I got unemployment checks afterward, because I didn't have to quit (as I planned) and they were too embarrassed to claim the firing was for cause.
What if, as a software engineer, you move job and start work on a pre-existing project? Do you review the entire codebase? I'm curious to know if there is a point or a situation in which you accept that you cannot know the whole codebase and, if so, how is this different from accepting code that has been produced by and AI coding agent?
Hi Hugh. It's not about "knowing the codebase" - that's only ever really possible on relatively small projects, preferably that you wrote yourself. Most of the time coders *don't* know the codebase - they can be far too big and complex to "know". It's about *confidence in the code*. Say you move jobs and come onto an unfamiliar large project. Can you have confidence in it? Well if it's production code that's been running successfully for many thousands of hours, then you can certainly have confidence in it. It's been battle-tested ad nauseum. The same thing is true for the many libraries and modules we use routinely. Say I decide to create a website using the Django web framework. How many lines of code from the Django framework do I need to review? Zero! Because Django is already used successfully in tens of thousands of web sites and web applications. It's already battle tested to death. So I can just go ahead and install Django and start using it. (And in fact the same thing is true with computer languages themselves. Python is written in C, so do I need to review all that C code so I "know the codebase" of Python before I start using it? Of course not!)
Say I take up a ticket on that large, unfamiliar project I just came onto. Actually all I need to understand is the effect of the change I am
making. It's only the code that I add (and it's effects on the broader system) that I need to understand, and only at the moment when I submit it for review. Another coder will test and review it, and then assuming it passes inspection it will get added to the project. At that point I can "forget" the update, because we have confidence it works.
Of course if you were brought onto a project which was NOT running successfully in production (perhaps you were brought in for that very reason - i.e. because it wasn't performing well) then you would not have so much confidence, and you would need to go bug-hunting.
The problem is LLMs don't generate battle-tested code. In fact there's no way to know if the code they generate has ever been used before. One might actually have more confidence in a code snippet lifted off StackOverflow because at least it has a number of upvotes (note this is a joke!). There are no upvotes on an LLM's code, or any indication of what influenced the output. They don't even generate the same thing, if asked the same question twice.
I hope that makes sense but let me know any questions.
Got a good laugh out of the typo, Gary: " ... METR study reporting noting poo results."
And thanks for the last paragraph of your article. Every day I see intelligent people send me AI slop masquerading as news stories, but too many people don't check them out. A lot originate on my eternal irritant, to which I have never subscribed: Facebook. The most recent one was about Rep Massie receiving a thumb drive with all the unredacted Epstein Files on it. This is very dangerous, especially as we approach elections. Trust is the worst casualty of all.
p.s. I couldn't get into the WAPO guest link; wanted me to subscribe and blocked the article.
So glad to read this. I studied computer science in the mid-1970s. Our bible was Donald Knuth’s The Art of Computer Programming. Here’s a quote from Knuth:
Computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty.
I would really like to match AI algorithms with human algorithms for elegance and beauty. Not to mention the lack of bugs. There’s going to be a tsunami of system failures as all this AI code gets implemented.
Incidentally, the very first thing we learned in computer science was Turing’s proof that a machine can never debug itself.
You think I don't remember the details of Turing's halting theorem?
I'm quoting you exactly what our lecturer said back in 1974.
Yes, I was very happy to see some random person who has some experience saying exactly what my intuition tells me. I'm way out of the game, but I still have my intuitions intact.
Let's wait and see how this AI coding goes, OK? Come back to me in six months, I'll start keeping a list of AI programming disasters. You can keep a list of the successes.
Actually, the very first thing our lecturer told us was, "Computers are stupid, they will do whatever you say, however dumb."
These days, people think the machines are really smart.
First, Fred didn't mention Rice's theorem, he made a claim about "Turing's proof".
Second: no. Both Turing and Rice's proofs show that no TM can determine a semantic property across *all* TMs, not that no TM can determine a semantic property of a *specific* TM, including itself--Turing didn't prove that no TM can determine whether it itself halts--there are in fact an infinity of TMs that can determine that of themselves (and an infinity of TMs that can't determine it of themselves), while there is necessarily *at least one* TM for which each of them can't prove it. It's important to get the universal quantifiers right, and many people don't.
And Rice's theorem only applies to *semantic* properties--many bugs, especially in type safe languages, are syntactic.
And again, *we are machines* (and only as computationally powerful as FSMs, not TMs) so Turing's and Rice's proofs apply to us as well. (See the Church-Turing thesis.)
Finally, the old saw about computers only doing what we tell them to is absurd in these days of machine learning and inscrutable neural networks--we've known better at least since Arthur Samuel's self-learning checker playing program that played far better checkers (even beating master-level players in 1962) than Samuel knew how to teach it.
That was my experience coding with the AI tools: prompt 1 -> bug 1 -> prompt 2 to fix bug 1 -> bug 1 fixed introduced bug 2 -> prompt to fix bug 2 -> re-introduced bug 1 and didn't fix bug 2 etc. etc. You get the picture. After several hours, I felt drained and I reminded myself I know how to code CSS, HTML, JavaScript . Switched back to my own brain. I totally agree with Tom Gracey.
HTML is not code, it is markup. I guess if you can "code" so can AI, but neither of you can do software development. (CSS and JS are not robust general-purpose software development tools.) Too many "coders" have been led astray into calling themselves "developers" (not necessarily you personally).
Even before AI there was a lot of really bad code out there made by mediocre programmers. If those mediocre programmers could understand why LLM generated code is usually sub-optimal, they would have improved their code themselves.
Instead, they now have a tool to multiply their incompetent output by orders of magnitude.
What do these rude obnoxious *hypocritical* virtue signaling children with their substance-free personal attacks think they are achieving? I've been on the 'net since my supervisor Charley Kline made the first remote connection, from UCLA to SRI, and never once has one of these pompous twerps caused me to change my behavior.
I'm a past coder, and have used AI to write code for me last year, and it did pretty well, despite trying to delete my data which I did not ask for, and in a few instances I was not fluent in two programming languages. But the time debugging it was a huge part of my effort. That reminds me, I need to track down how to stop paying a monthly fee for it. Both the nature of my job changed and the security issues have kept me from wanting to trust it for enterprise level coding tasks.
I guess one can use AI for automatic review of code; models can point out parts of your code that do not adhere to typical patterns.
This might be a bug… or a false positive. And I can deal with false positives.
Another is for researching causes of bugs before manually debugging. But if the model doesn't get it right in the first answer, it is not likely to get it right with later answers.
Both applications seem non-harmful because there is no AI slop production involved.
Agree that in constrained reviewing tasks it can add value, but the value is more likely to be incremental increases in quality (if used right) rather than "10X productivity gains" that the C-suite thinks its getting.
I used it to review some moderately complex code the other day. The latest and greatest, Opus 4.6. Read the entire 2,000K line file into a clean context with plenty to spare, about 77% free. Asked my Claude Code (with a careful, detailed, prompt) to review the code and existing test cases and create new test cases identifying bugs or untested branches, being careful not to create redundant cases.
Pros - it did identify two cases not covered, that revealed legit bugs. This was useful to me as I'd reached the point where I was too mentally tired to identify further gaps.
Cons - it spent the rest of its time generating redundant test cases either for the two cases it had already found or for the other many test cases that already existed.
Point being, it's a useful tool that I appreciated, but even on an incredibly narrow and constrained task its competence is vastly overrated, and it doesn't "reason".
Don't get me started on what it does on a small 10K multi-file repo, or a 100K larger repo, or a project in the millions of lines spread across repos.
The issue is also that this is heavily, heavily subsidized right now. If it cost like 20×, 30×, or 40× as much … I would drop it even for my rather limited use.
Development with AI (LLMs), I think, is always a very bad idea in all but select circumstances (e.g. sandboxed mockups): it generates enormous amounts of technical and cognitive debt very fast.
But more importantly, AI development is even now, despite subsidies that are ruinous for those companies, still very expensive. It rapidly burns through token since the AI has to read in a lot of files into the context for implementing a feature.
It really gave this type of development a try and ended up with a bill of $400 for a clunky, extremely buggy, and unmaintainable Frankenstein application. Imagine if that cost me $8000 instead!
Software engineers have always been like sorcerers to the non-technical, someone that produced magic and without any way for them to verify what they said was correct or not. At best, they had a technical oracle who could advise them.
Not anymore. The non-technical vibe coder has achieved sorcery without understanding what it is doing. Hell, probably a good 50% of actual coders or more really didn't have a great understanding of what they were doing, just copied and pasted, as human LLMs, hoping for the best, feeling when it seemed right, and without any ability to reason about it.
So of course they think they have conquered coding, logic, and reason, through vibes and being a word-cel. They can pretend to be shape rotaters now.
I think the use of this example is forgivable because it's not an area that he has expertise in, and the public discourse is shouting that this is a viable use case for AI. That's the "prevailing wisdom" at the moment, let's say.
Those of us with experience making software, however, know that AI is turning out to be a net negative and the discourse about that is finally catching up. The prevailing wisdom will be set straight and people won't be able to use it as an example of a viable use case.
Could you please leave, and go to Reddit, Facebook or X, if you cannot keep a polite tone, and insist on offending people? Or maybe try discussing on HN, let's see if others let you behave like this there. Thank you.
I get what you're saying. "(AI) coding is a practical use case" should come with the caveat that AI coding is not an automated replacement for human developers because AI requires much oversight to prevent bugs and security vulnerabilities.
I don't think that Gary wants to say that AI coding is solid, he has always said that it's good to create prototypes and drafts that should be then refined by human engineers
Which somehow resembles the 'rapid prototyping' (I think it was labeled a bit differently) silver bullet of the 1990s. That failed for the same reasons that code generation at that time failed, mostly because the stuff could don't be maintained.
The skills of experienced coders would be more convincing if code bases weren't riddled with bugs, that QA has to be run on every code change to make sure the code doesn't break, that platforms including OSs don't need constant updating to fix bugs (and malware exploits), and periodically there are major outages often claimed to be due to a "coding error".
At the other extreme, I recall talking to a coders for a military comms system telling me that just to alter a line of code needed documenting and paperwork that was reviewed by senior people before being allowed as a change. No doubt this excessive [rocess was a CYA and means to track where erorrs occurred.
AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language. Errors will occur. GenAI will make errors too, but by not hoping to have the AI do a complete app, but by doing it piece by piece and running unit tests, boilerplate code can be created more quickly, and mistakes corrected early. Thhe idea is to keep a human in the loop and use the AI to increase the production speed, even try out different approaches, saving cognitive effort.
Humans make miskes in all sorts of endeavors. Even this economic analysis may be a mistake, so I will wait for other economists to weigh in before accepting its claims at face value. [c.f. the famous debt/GDP growth error by Rogoff that was due to a spreadsheet error!] How many news and technical articles were just wrong written by "experienced journalists and reporters?
There is also definitely an issue of lost opportunuity costs, where investments in datacenters has starved fiunding of potentially more attractive projects and businesses which has been raised by experts before.
Marcus, and others, is correct to point out where GenAI fails badly. But that shouldn't mean calling the whole enterprise a failure. Marcus has said he hopes for real AGI. I do too, but at far lower cost and resource use than we currently do. I hope LLMs are like Newcomen's atmospheric engine that was very inefficient at converting the energy in coal to work, and was obsoleted by Watt's steam engine that was far more efficient and was the main mechanical method of delivering work for manufacture and transport.
So while we may be moving along the hype cycle from the peak of inflated expectations to the trough of despondancy, I expect genAI will find its plateau of productivity. Eventually it will evolve to something like AGI, but not necessarilly to human level AGI with our foibles, but something orthogonal, like our machines that fly, but not like birds.
Give me a fresh CS graduate over an AI any day. The fresh CS graduate can learn and improve. They can explain what they are doing. They can be tasked into isolated areas with good examples to follow.
The AI coder is more like an extremely evil coder who gets hired to sabotage your code, but make it look like it isn't sabotage. They will be productive enough you stop questioning them, and then sneak in the backdoor to kill you.
> AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language.
Hard disagree here.
Among the differences are that fresh graduates and experienced coders can learn, they don't need to be reprompted with the same forever, and they can reason so they don't generate probabilistic slop that is just a superficial facsimile of following instructions, e.g. the example I gave in a reply higher up the comment thread when even with a completely self-contained 2,000 file, a clean context, and explicit prompting, Opus 4.6 can't stop itself from generating redundant test cases that all test the exact same scenario.
I would be more apt to compare it to an incompetent but very experienced developer. Anyone who has worked with this type, full of arrogance and things that seem ok, vs. a new gras who cannot do as much harm, knows the difference. As the models get better, they get better at fooling reviewers with things that look right, or are too complex to reason about, so you are left with just hoping your test cases are sufficient.
> AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language.
Absolutely. 100% agree. As in “fresh CS graduate” or “an experienced coder moving to a new language” are TIME SINKS, if they are on my team then I'm moving slower and together we do less than I could do alone. Did that a few times, just recently.
And that's how it works with AI, too.
That part is the same.
Now, after half-year or maybe one year, with a good CS graduate or good experienced coder moving to a new language I would get return on my investment: we start, finally, be more productive then I was alone. With AI… this part somehow, never arrives and this blog perfectly explains “why”.
P.S. But AI is pretty decent with writing one-time throw-away scripts, I'll give you that. And you can use that. But trillion dollar investments to write some throwaway scripts sounds a bit too much, if you'll ask me.
As for coding. An LLM It might be useful in boring, non-critical, repetitive tasks, and for quick prototyping. But the problems will always remain, no matter the amount of plugins or the improvements of the LLM wrappers(Agents). The LLM gives you statistical data, an average of all the code it has been trained on. Basically, it cannot ”invent” something it hasn’t seen before, it cannot learn to code from a programming language documentation without examples. The average of the code it gives you, is neither the worst or the best. And average code is filled with bugs and other issues. This will never improve due to the very nature of the transformer model LLM is built upon. If you want code, filled with bugs(behavioral, subtle bugs(hard to detect and can linger on for years before they explode) rather than naïve bugs), is non-scalable, code with bad architecture and design, LLMs will help you with that very efficiently. For code that needs serious attention, LLMs still require a senior human guard. I don’t see that can ever change with the LLM transformer model, no matter how thick and shiny the wrapper becomes.
With respect, great that it works out for you. Great too that it doesn't work out for others or development teams because each has *different* goals, *different* applications in different domains, and *different* workflows.
Some software developers have to architect, and develop and support software with complex features used by thousands or millions of paying customers. Other software developers are compelled to follow company guidelines (as in my own company). Yet many others work as independent consultants or they're just experimenting like retired programmers.
The term "software" is very highly nuanced. These differences must be respected. And there is a lot of crappy software out there.
I have been programming for about 40 years and managing programmers for the last 25 years. For the last half year I used Claude Code every day. If you are a very disciplined software engineer that works with design documents and requirements up front and then do testing every step of the way, you now can do magic and go 10 times faster than you ever could before. For all of those in this thread that are no active programmers and haven't used Claude or Codex I can say that you are debating the taste of a dish you have never eaten. The reality on the ground is that these tools are transformative for people who already know how to engineer software properly. They are not magic wands for the undisciplined, and they are not toys for the experienced. They are power tools, and like all power tools, they reward skill and punish carelessness.
you are welcome to comment here - for now - but please watch your tone. i got complaints on some of your recent posts from multiple people and if they continue i will have to ban you for a bit.
Noted ... but consider that my comments received numerous likes and people attacking me for my "tone" say things like "Oh please, Shut the Fuck Up; you're boring as hell. Are you working for OpenAI or Anthropic? Why are you shilling for them?" ... so they may not be the most objective or reliable sources.
Agreed. I am also a skeptic on the excessive claims around LLMs. But I use it judiciously in my work (which involves development and data engineering) now because it does have practical use cases. It's not perfect, it's not effort-free, it's not a one-size-fits-all solution and it needs well structured inputs and proper review of its outputs - but the same could be said of my dishwasher, and I get plenty of use out of that.
The use case you cited of translating code between languages is particular pertinent since the best context of all for a generative model is an already solved problem where there is no ambiguity. In a similar vein, I've also used it to very quickly understand the nuances of other people's codebases, since parsing thousands of lines of unfamiliar code as a human is laborious and doesn't play to our strengths - but getting the model to explain and cite allows to me to skip the fluff and validate the logic quickly.
It has undeniable economically useful applications - it just isn't on the level they claim (since they are perversely motivated) and if it had been sold as an expensive but useful tool for developers I suspect nobody would have cared or noticed outside of tech.
And 100% there is a reckoning on the horizon for businesses that over invest in the tech and under invest in their staff during this stage of huge subsidisation by hyper scalers (the pre-enshittification stage).
What this tells me is about what I expected. Using these tools will be helpful, but they will not be usable by novices. The current hype is that software is obsolete because users will be able to use AI to solve their own problems with AI generated code. I was deeply skeptical that these tools can be unleashed without professional supervision. All of the current selling off of SaaS companies is predicated on a myth that their products will be dumped. More likely, those products will just be more efficiently maintained and upgraded using AI tools and the clients will still pay for them.
The “current hype”? More like “decades old hype”! Remember the LAST time something like this was happening? AI was supposed to ensure that knowledge of software development would no longer be needed! Exact quite is “If the Japanese succeed in their goals, the only language you'll need to know to talk to a computer is Japanese”. The hope, back then, was that parallel computing and AI would deliver that. And people would finally, finally, finally, be able to tell computer what they need and computer would program itself. Yeah. Read the “The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to the World” book, it's not much different from Sam Altman promises. Well, the big difference is that it's 1983th year book, not 2023 year book, but who's counting? We would see more such book in the coming years.
LLMs certainly can be used by novices in writing, assuming the novices are receiving training in fact checking, source checking and all the other stuff that novices have been trained in for god knows how long. Why does everybody think that if LLMs hallucinate they are broken? It's just a trait, born of solvable with a modicum of sense and training
The question is not whether LLMs are broken or not. The question is if they could replace a software developer. Not HELP, not SPEED UP, not “marginally improve”. But REPLACE.
If that doesn't work then there are no justification for trilions of dollars spent, it's as simple as that.
P.S. for an example of how widespread and growing use of "AI" (I quote it because LLMs are not intelligent) in code development is problematic, see https://www.youtube.com/watch?v=bZJ7A1QoUEI
It seems like you assign some kind of deep moral valence to differences of experience or opinion in this area - as opposed to just stating that you have a different experience and set of observations and simply sharing those dispassionately. I'm trying to understand why you sound so angry.
UGH that was with model n, model n+1 that has 3% higher score on a benchmark that has test set completely contaminated will solve everything. Now just hand over 300B you damn luddites.
Since it may not be exactly clear that was sarcasm.
So AI contributed "as little as zero" to the US economy in 2025.
How do you know it's not negative? How much time has been wasted? How much money has been wasted? What could have been done with that time and money and focus and energy, if it had been spent on something useful, something constructive, instead of trying to destroy all jobs, destroy the education system, destroy people's minds and turn them into ChatGPT zombies?
How about the time spent fixing absolutely insane AI hallucinations? I've spent two weeks of a three-week editing job just trying to fix fake references generated by ChatGPT. Most of my work in 2026 has so far just been fixing AI disasters. Productive much?
I've commented and written many times that AI is going to bring me a lot of work. The more they realise human oversight is needed, the more they will have to call on the professional editor. I've said, business is going to be good.
My paid contract is to edit academic papers, which I've been doing full-time for 12 years without a hitch. My first assignment this year has been a total AI disaster that has taken weeks longer than anticipated and messed up my schedule badly.
It's very hard to prove the negative, that a certain reference does *not* exist, especially when the authors and journal appear authentic. Editing AI product is more time-consuming than any other work I've ever done. So I am raising my prices for AI clean-up.
Good work, Fred! I bet the price tolerance for AI clean-up is high, higher than most people might think. By the time they come to you, the project is in crisis.
I expect business will be booming!
Your biggest challenge will be to keep from going completely mad in the process. I wish you strength!
"I think it's pretty likely the entire surface of the earth will be covered with solar panels and data centers." ~ Ilya Sutskever
This has to be one of the bleakest, most deluded takes circulating in the industry - beaten only by the pure wishful thinking these numbers lay bare and the painfully obvious shortcomings of generative AI and conventional hardware.
If your model requires a reactor, build a better model.
Where did he say this? I'm interested as it is such a(nother) horribly stupid, naive thing to say. Even without fossil fuels such energy use would heat up the atmosphere faster than greenhouse gasses are doing already. Reference? I can believe he said this, but I'd like to be sure.
Hinton (being a main source for this kind of naiveté) has quite a bit to answer for.
It seems that comment was made in the context of him expecting ASI and that an ASI would cover the world in solar panels and data centers and that it was thus important for humans to make sure ASI is benign for humans. So this is not so much a wish but a Terminator/Matrix style of expectation/fear based on the inevitability of ASI.
Requiring an account doesn't mean that it doesn't work. The link is free with a non-paying account, which is how I was able to read it. (I canceled my WaPo subscription when Bezos scotched the editorial board's Presidential endorsement.)
You do understand this is classic landing page data harvesting - i.e. your email. You will then be plagued with offers and other monetizable stuff to help turn the WaPo's finances around? A [gift] link should be a no strings attached link to opens the content to read.
Non sequitur. And don't insult me by snarking about my knowledge ... I've been on the 'net for over 50 years, hold multiple network patents, and have a mention in RFC 57 from when I worked on development of the ARPANET for the UCLA Computer Science department ... there's very little that I don't know about this technology.
All good. Then please accept that I do not want to hand over my email just to read an article for good reason. As you are aware, Substack is locking down posts that one has not paid a subscription and using the "read in the app" to do the same, probably worse given the access smartphone apps have on your data.
Agreed. New York Times is at least better on that account, although their coverage on other issues maybe sucks worse. Anything to do with transgenderism in particular -- their description of the shooter in Tumbler Ridge Canada, Jesse Strang, insisted on referring to him as a "she" and "her", and gave lip service to the claim and insane idea that he [a male] was "transitioning to female":
QUOTE; New York Times: On Tuesday afternoon, Jesse Van Rootselaar, 18, grabbed two firearms from her home and, the authorities in British Columbia said, killed her mother and 11-year-old brother. Then she traveled a mile to the Tumbler Ridge Secondary School and killed five students and one educator before turning her weapon on herself.
.... Ms. Van Rootselaar was raised a boy and began transitioning at least five years ago to female, according to the police, which identified her as female. UNQUOTE
Batshit crazy to talk about "transitioning" to another sex. They might just as well have accepted a claim that he was "transitioning to a penguin."
But here's a share link that won't oblige you to sign-up there at the Times 😉🙂:
I have been involved in the 'responsible innovation' conversation at various levels of involvement in nanotech, synthetic biology, robotics, neurotech, quantum tech and AI, back when it meant algorithmic design and big data. The problem is 'The Economy of Promises' first coined by Physics Professor Richard Jones in 2009 in relation to nanotechnology (where I met and worked with him). The incentives from the large government research and academic funding agencies, through to venture capital and other mainstream funding only reward hyper inflated promises about changing the world, so that's what they get.
I wrote about it for the WEF blog on What we can learn from the past for new tech, in 2017.
"The way funding works is that in order to get the money, scientists and businesses have to massively exaggerate the potential benefit of their ‘ology' - ‘an end to hunger’ (GMOs) ‘electricity too cheap to meter’, (nuclear) ‘the end of disease’ (nano) - the media love it, funders get excited and the money flows.
But this “economy of promises” is just another form of fake news, with potentially damaging repercussions:
The short-term reality can’t possibly live up to the hype and the trustworthiness of the technology is tarnished. For example, the failure of the US National Cancer Institute’s 2004 aspiration to eliminate death and suffering from cancer with nanotechnology by 2015 might have left many a little disappointed.
Regulators have to start early to consider legislation around the risks and hazards of a new technology. The only place they can start is with what scientists and businesses say they will deliver. Those developing regulation in the life sciences have also found, with hindsight, that this focus is problematic. The consideration of nano-related risks and regulation driven by the hype, compounded by the thrall of the ‘ology’, was confusing and distracted from exploration of the reality of risks and hazards.
So what is the obsession with creating life-like robots, linked to the hype about ‘The Singularity’ where within 30 years robots and people will merge and we all become ‘post-human’, doing to our ability to develop robotics for useful purposes? What will be the result of the hype about the precision of CRISPR and gene editing be when, it becomes clear it is not a panacea for every genetic disease? Are we wasting time debating the ethics of science fiction when we could be discussing the not inconsiderable impact of the reality."
The real problem here they have too much money which they can spend shovelling into the idea, puffing it up, inflating the market and the influence they have got in making it look like a solution to the big problems we face which means we swallow their bullsht.
Such a shame we couldn't go the helpful practical route in the first place and focus on solving human needs and not technological progress for its own sake, which is notoriously hard to tell the fact from the fiction in every tech.
The (mythical) Singularity isn't about a "merge", it's when AI will purportedly exceed human intelligence, become capable of refining and improving itself with no limit in a positive feedback loop, with the future of the world at that point becoming completely unpredictable and opaque to us--that's why it's called a "singularity" analogous to how the physics of a black hole is like dividing by zero and thus beyond analysis. There's a lot wrong with this story, but it has nothing to do with robots and humans merging--humans become superfluous in this tale.
However you slice the numbers between imports and final outputs, it’s obvious that hyperscalers and neoclouds are spending massive amounts of money on data centers in the US, which does contribute to economic growth. What is much less clear (and probably untrue) is that the final generative AI product is doing anything positive for productivity or growth. IMO *that* disconnect is what’s going to cause problems in the very near future
This is exactly my reaction. Capital is being spent, right? Contractors are building things? And paying their employees a wage? Perhaps this is money being misspent, and perhaps it will end up being a net negative in the future, but by the normal method of measuring "economic growth", huge construction projects contribute in the present.
Contractors *aren't* actually building things, or at least a whole lot less than is being claimed, as much of the data centre "growth" being reported is still on paper.
One major user of data centre space, Microsoft, has been taking its foot off the gas. Perhaps they know something...
By this standard, Keynes’ observation about paying people to dig holes and then fill them in again also “contributes to economic growth” in the sense that GDP goes up.
That is NOT AT ALL the same thing as productive economic activity.
Sure it is: The hyperscalers pay construction companies to build the data center shells. This means they have to pay the contractors/subcontractors, buy concrete, wood, metal, etc. They buy chips from Nvidia/AMD/etc, and the employees of those companies make money. All of those people take their earnings and go buy stuff at the store, pay their mortgage, etc. It is definitely productive economic activity!
In fact, Keynes's whole *point* about paying people to dig ditches was that even though it seemed like pointless busywork, it would be a valid method of government stimulus during a recession/depression when aggregate demand falls. And paying people to do *something* helps give people more purpose than just mailing them checks.
Two things can absolutely be true at the same time:
(1) The boom in GPUs and data centers is stimulating the economy
and
(2) The ultimate end product of that investment (I.e. generative AI/LLMs) is not useful enough to get ROI on that expense, and that's not even counting the numerous negative societal effects
I'm also bearish on GenAI and worry this bubble is going to blow up, but if we can't honestly assess the situation and throw Econ 101 out the window we're going to sound like hysterics and lose any credibility.
Well stated, thank you. It's unclear if the WaPo article claims (2) or if it claims that (1) is not actually true because that is just still on paper. Or a combination of both.
I skimmed the article and it seems like a very technical (pedantic?) argument that because so much of the upstream materials like GPUs are imported from foreign sources, which is subtracted from some calculations of GDP, that it's having like a net zero effect. I'm not really sure it works that way. Even for GPUs that are manufactured overseas, a lot of the money is going to US companies like Nvidia, AMD, etc.
I agree with you right up until the mention of Econ 101, which desperately needs revision, absent which throwing it out the window would actually be beneficial.
BREAKING: First semester college course for beginners does not capture full nuance of complicated field, details at 11.
Seriously though, just because the basics don’t cover every conceivable corner case in a modern economy does not mean we should throw out the fundamentals. You may be shocked to learn that there are also oversimplifications in “Bio 101” and “Physics for humanities majors”
Which really happens. In The Netherlands, a bunch of lobbyists wrote a 'national AI plan'. They (wrongly) suggested as written at the behest of the government and in it they say that they wrote it together with an LLM. Horrible piece of cheap nonsense, trying to to get money by startups and VCs and some policy wishes from ASML.
"I don’t know that Generative AI was literally a scam, but the people selling it have tried to sell it as if it were tantamount to artificial general intelligence, when it’s not."
In every iteration, electronic parrots based on large language models passed off as 'artificial intelligence' have always been a scam.
Or as I said it last August-
'AGI- A fraud wrapped in a scam tied to a bamboozle.'
"No amount of laminated computation will beget emergent properties of intelligence, let alone awareness, let alone self-awareness."
To be fair, the scope of the WP article is relatively narrow and doesn’t seem to bear the weight of the conclusions being drawn from it. The article is largely silent on the alleged productivity gains, economic effects or use cases of LLMs. Rather, it focuses purely on the distribution of the value from AI investment, arguing that a substantial share of the value accrues to foreign producers rather than the US, given the significant proportion of imported computing equipment and components embodied in AI investment to date.
The AI models we have now, with their staggering, absurd costs to train them, should be seen as expressions of totalizing faith rather than a bold infrastructure project (like AI companies try to sell them).
More like Egyptian pyramids, not like Roman aqueducts.
What we gained and learned from those models, I'm sure could've been achieved much cheaper. Without getting into this frenzied race, gambling the whole economy on it, and ignoring all the massive collateral damage.
In the end, those models even lack the grandeur of the pyramids… Instead, they are embarrassing and brought out the very worst of humanity.
I've spent the last week bouncing between things written by people whose opinion I respect on how Claude Code is amazing and game changing and things written by other people whose opinion I respect on how it generates crappily-written non-secure slop and I'm so tired right now. It did help with a little project but I'm not ready to generalize to 'rewrite Salesforce' from that.
Seems difficult to draw strong conclusions when the article specifically points out the economic data are complicated to measure, leading them to be potentially misused for any narrative:
“It’s clear that the huge spending on AI is adding to the U.S. economy, but the available economic data doesn’t neatly capture its effects. The debating economists and the slippery data suggest that if the technology does start to reshape the economy, it may be challenging to detect and clearly measure. That may leave political and corporate leaders to *choose the numbers that fit their preferred narratives* on how AI is changing American life and work.”
This was a textbook Microsoft strategy of embrace, extend, extinguish applied to an nascent industry that looked as though it could threaten their os monopoly.
They inflated openai's initial valuation with a ton of compute credits only spendable in azure. Forcing openai to do repeated mass funding rounds to keep the lights on while they use those credits.
They gassed up the media and legal realms with talk of how world changing the technology would be.
The talked of making Google dance.
They flopped on every single attempted rollout of their own ai integrations, mostly passing people off in the process. Copilot is still hot garbage.
They massively inflated nvidia purchases leading to a tragedy of the commons scenarios for big tech companies which forced them to buy under the assumption that if they were behind it would worse than if they overspend. Now they have accumulated massive debt with more to come and they are blowing through their free cash.
Microsoft hasn't been first to an industry since os and every industry they have entered since has been a wedge to protect windows not to actually compete well within that industry. They are not innovators, they are smotherers.
Will you PLEASE stop saying "coding is a practical use case"? This is the third appeal I've made on this subject. (Do you read your comments?) If you want bug ridden code with security issues which is not extensible and which no-one understands, then sure, it's a practical use case. Just like if you want nonsensical articles with invented facts, then article writing is a practical use case. But as I've pointed out already no reputable editorial is now using LLMs to write their articles. Why is that? Because it obviously doesn't work.
Let's face it the only reason you're saying "coding is a practical use case" is because you yourself don't code, and don't understand it. I can't see another reason why would assume the problems experienced in other domains somehow don't apply to coding. Newsflash: they do. And software engineering definitely doesn't need the slop any more than anyone else. So I hope this is my final appeal: please stop perpetuating this myth. If you want more information on the problems of using LLMs to code, then I can talk in great length about it - feel free to reach out. Thanks
i started coding when I was 10 years old, in languages from 6502 Assembly to C# to Python to Swift and many others. Many of my friends code professionally. I fully agree about the security risks and maintenance risks and have written about them here multiple times (eg the essay LLMs + Coding Agents = Security Nightmare). I have written about the METR study reporting noting poor results. But (a) i think you misrepresent what i have written about coding as a whole and (b) think it would be intellectually dishonest to say there is no value for coders, especially given reports on the latest iteration of Claude code.
Hi Gary - thanks for your reply. Firstly just to point out I wouldn't comment at all if I didn't already particularly respect your stance on AI in general, and more importantly the effort you've been making trying to counteract the "LLMs will produce AGI" narrative. In fact I'm pinning a lot of my hopes on you (and also Ed Zitron, whose blog is the only other place I'd comment - if he'd not switched comments off). No doubt there are plenty of other people saying the same thing (one just needs to look at the comments here, for example) but I don't think they have quite the reach or influence as yourself. (We need a hero Gary, and you're it! And I'm only half joking...)
So that's the only reason I write here - I'm otherwise really very busy (creating software!) and feel I need to avoid getting caught up in any lengthy online debates. Therefore I'm just going to write this one response and hope it covers at least some of the objections made by commenters to my original message. Apologies for not responding to everyone individually.
First I'll address your points (a) and (b):
"(a) I think you misrepresent what I have written about coding as a whole":
It is quite possible I have mistaken your position on it - but if so this was not deliberate. I was merely responding to the number of recent occasions I came across where you appear to have singled out coding as a particularly appropriate use case for LLMs. However, if I did indeed mistake your stance on this, then please could you clarify it.
"(b) [I] think it would be intellectually dishonest to say there is no value for coders, especially given reports on the latest iteration of Claude code.":
I didn't claim there is no value for coders. What I objected to is having coding singled out as a particularly appropriate use case for LLMs. There is clearly value in LLMs for any field, since for example they can quickly return a specific answer to a specific question, and this is much faster than a traditional document-based search. But this is true for everything from cake recipes to Anthropology. Similarly some commenters have pointed out that LLMs are great for getting an explanation about a complex code snippet. This is true, but once again is not restricted to coding. You could post a complex token set of any kind, including financial statements, legal documents, poetry (yes, poems can be complex and difficult to decipher!) - etc.
re. "reports on the latest iteration of Claude code" - I can't help feeling that a great many of these "reports" are more hype-driven rather than objective. (Ed Zitron agrees: "You’ve probably noticed in the last three months that everybody suddenly started talking about Claude Code, and you’d be forgiven for thinking there was some sort of massive product update rather than, as it turns out, one of the most aggressive media and astroturfing campaigns that the tech industry has ever seen." From here: https://www.wheresyoured.at/premium-the-haters-guide-to-anthropic/#how-anthropic-rugpulled-everybody-using-claude-code-deceiving-investors-the-media-and-its-users).
(Note there is actually some really quite questionable engineering that went into Claude Code itself: https://www.youtube.com/watch?v=LvW1HTSLPEk - one amusing comment on that video: "it took me 12 minutes into the video to comprehend that react has in fact been put into a terminal")
So far I've pointed out the following usages are not unique to coding:
(a) searches (I call them "tokenised searches");
(b) summarising complex token sets;
Unless I am mistaken, we are then left with (c) the question of generating code for actual use in software applications.
On this subject, firstly I'd point out the following:
"Coding" - as in Software Engineering - is actually logically isomorphic with any other engineering discipline. i.e. you could choose to code up any engineering schematic instead of e.g. drawing it on a drawing board. For example you can create a Python script for Blender and have it render a 3D plan for your house - or whatever. You could do this with engineering plans of arbitrary complexity including multi-component systems such as cars and aircraft. (I personally will not be volunteering to be the test pilot in an aircraft designed by an LLM!) We (humans) choose to draw plans first - but that's just because we find them easier to visualise. But if LLMs are particularly good for Software Engineering, they should be particularly good for these other engineering disciplines too - so instead of singling out coding, we should be able to just say "LLMs are an appropriate use case for Engineering".
That's an argument against singling out Software Engineering WRT point (c), and I'll now add just 3 points corroborating the view that Software Engineering is not an appropriate use for LLMs. I could provide many more (let me know if you want me to do that) - but this message is long already, so I'll just write down the first 3 that come to mind:
1. I mentioned that coding is actually Engineering. I think it's important to regard coding as Engineering because it makes many of the problems with using LLMs to do it instantly more obvious. Engineering is not founded on semantics, it is founded on logic. However, LLMs are good at semantics and not at logic.
LLMs have ingested a lot of code snippets from Stack Overflow, and tend to be good at rehashing those code snippets. That doesn't make them good at generalising - on a logical basis - to a much larger codebase. Humans, when they post on StackOverflow, tend to whittle the problem they are facing down to as small a representation as they can, in order for it to be appropriate for a discussion forum. Then when someone posts a solution, they take that and generalise it to their real situation. They don't post their full codebase.
As a result "successes" of LLM coding seem to be restricted to bite-sized apps which rehashing StackOverflow-type snippets can have a hope of generating. Again, based on semantics, rather than logic.
(I'm sitting in front of a large, complex codebase right now - and the thought of setting an LLM loose on it just makes me laugh!)
2. While we're on the subject of StackOverflow, prior to the advent of LLMs one could have created an app "in record time" by simply copy/pasting code snippets from StackOverflow. i.e. you don't bother to check over each line to be sure of what it does - you just go with whatever the solution is, without attempting to understand how it works or whether it's really the best solution for your particular situation. You stitch all these snippets together and voila! You have your app built in record time. Needless to say, no-one would ever recommend doing this.
The point is, there has always been a trade-off between the speed of development and quality of engineering (confidence in the code, robustness of the app etc.) I don't see LLMs as either changing this trade-off or shifting the needle (greater quality in a shorter time), because they are probabilistic and can't be relied upon to produce the best solution - or even a correct solution - every time. So you're going to have to pick your way through every single line it generates in order to have the same confidence you would have if you wrote it - and this is unlikely to save time because understanding someone else's code is always more difficult and time-consuming than writing it yourself. When I hear people say it is "making them 10x more productive" at coding, I think, "and also 10x as unsure what you've actually produced".
3. You'll also need to correct it when it does something you don't want. Now this is pretty interesting, if you think about it. Imagine you provide an LLM a prompt, and the LLM produces something but not exactly what you want. What is the advice on this? "Provide a more specific prompt!" Ok, so then we write a more specific prompt - the results are better, but it still falls short. What now? "Keep making the prompt more specific!" Ok but wait - eventually won't I be supplying the same number of tokens to the LLM as it is going to generate as the solution? Because if I'm perfectly specific about what I want, then isn't this just the same as actually writing the solution myself using a computer language? Indeed, isn't this the purpose behind computer languages in the first place? Note that in computer system development we've already addressed issues related to how specific you need to be to define a system: that's the "level" of the language; for example C is a lower level language than Python, and so would be a more appropriate choice if you need to define specifics beyond that which Python can accommodate. This "level" works all the way up to website builder applications, which are really quite high level - for when you want e.g. a payment processor integration, but you don't care about how it looks. If you say to an LLM "build me a payment processor integration" then the LLM needs to fill in the blanks and actually design that integration. Yes, you're asking the LLM to be a designer (does that seem like a good idea?) So in just the same way as a website builder, you're going to get a payment processor integration, but with no control over how it looks. That is, if it works. In the website builder case, it will definitely work. So why bother with the LLM?
Note one reason I'm taking time to try and explain my perspective on this is because I was worried about the degradation of software development long before LLMs came along. Now I think we've really hit a new peak of silliness on a scale I never imagined possible! So please understand my original message was genuinely an appeal, and not an attack. Thanks
This gets to the root of the problem as I see it. Coding isn't a thing on its own. It is the end of a long chain of reasoning and decision-making. So even if an LLM could "code" perfectly (which it can't and won't do anytime soon), there are still a huge number of things that a developer (particularly a senior one) does that aren't "coding" per se.
Think about hiring a contractor to write some software. You have to give them a spec, and unless you want the project to eat unlimited amounts of money and time, that spec will need to be quite detailed. It must explain in detail exactly what the product should do, give some amount of explanation of how it should look, what important error conditions need to be handled, what the acceptance criteria are and so on.
A full specification for a non-trivial product can run to hundreds or thousands of pages and take months to write. Even with a great spec it is common to go back-and-forth many times before final acceptance.
Current coding assistants can't do any of this, really. They help some people avoid doing what they consider "drudge work" and that's about it. The idea that developers are going to be replaced by this generation of AI is hogwash.
Which isn't to say the absurd hype-level won't lead a lot of companies down the garden path to ruin. It will.
He said "coding is a practical use case", not that it is the only practical use case, so arguments that it's not alone are *non sequitur*. And if one looks at who is making serious use of LLMs, who is paying Anthropic et. al. for tokens, coding predominates. So a demand that he stop saying that "coding is a practical use case" is not well founded.
> As a result "successes" of LLM coding seem to be restricted to bite-sized apps which rehashing StackOverflow-type snippets can have a hope of generating.
This is not true.
> prior to the advent of LLMs one could have created an app "in record time" by simply copy/pasting code snippets from StackOverflow.
Nor this. An example is the translation of the Ladybird browsers from C++ to Rust in weeks rather than months. Another example is https://github.com/ecto/loon
You should consider combing through Hacker News to see how people are actually making successful use of LLMs (there are also many forms of misuse, abuse, and failure). It would be time consuming and I wouldn't blame you if you don't, but without doing so your statements are not grounded in fact.
Note: I am not endorsing the use of LLMs in coding, for a number of reasons including the economic and environmental impact of this technology. I am only addressing the facts about the use of that technology, and one of the facts is that coding is "a practical use case", for a properly understood notion of "practical".
P.S. Speaking of "practical", this just landed in my inbox:
https://events.zoom.us/ev/AlSncoNtWRs379zs7DhmO7Vx06aIhJTLKjHgPalxfOogi5EIxLaz~Arw-Hs_hU-WWGS-xxmS1nPCFV1l6KNfiMzJ2ccfiMbmHOCI06hABafBPBQ
"ACM TECHTalks: A Practical Introduction to Agentic Coding"
Like it or not, "coding is a practical use case" of LLMs.
1. "coding is a practical use case" I was referring to the fact Gary has singled out coding as an example of an appropriate use on multiple occasions, and not the literal meaning of this one particular reference.
2. "And if one looks at who is making serious use of LLMs, who is paying Anthropic et. al. for tokens, coding predominates." - The fact that lots of people are trying to use LLMs to code does not make it a good idea, or mean they are successfully building high quality, robust software.
3. "This is not true". You say it's not true; I say it is true. All you need to do is download Claude Code and try and build anything with it, and it is immediately apparent this is the case.
4. "Nor this." Actually this is very much the case. We software developers very often pull chunks of code from various locations - not just stackoverflow. Very often they are chunks of code we wrote ourselves, that we then adapt to the new system we are inserting it into. This is great, because we don't need to make an effort to understand the code we're inserting - we already understand it, because we wrote it. If we lift code from somewhere like stackoverflow - or take it from some open source module on github etc, then it's very important we make an effort to understand how it works, because it might not be completely appropriate to the specific location within the system we're inserting it into, and may therefore work as expected in some circumstances but not in others. One can indeed not bother to try to understand chunks of code that one inserts, and pay the price of not being sure that code is going to work correctly as a result. As I mentioned, the point I was making is that there's always been a tradeoff between speed of construction and confidence in the robustness of the solution (I am sure you don't deny this is the case). Furthermore I think that inserting code chunks without checking you understand them is actually very similar to coding with an LLM - the only difference being the LLM generated the code chunk instead of you finding it online somewhere.
5. "You should consider combing through Hacker News to see how people are actually making successful use of LLMs" - the problem with this is there are really a lot of hype-driven stories out there that are basically made up. I've caught some that are obvious - e.g. see my comment on this post: https://substack.com/home/post/p-185469925 - which then makes me quite sceptical of many of the others. I'm not really sure why this kind of fabrication has become so prevalent - I find it very strange - but there's certainly a lot of it going on. At the end of the day I'm going to trust my own experiences actually trying to use these tools, and not stories about them that I can't verify.
Sigh. I've martyred myself in this cause. I got fired from a job in 2009 because I refused to incorporate 3rd party code from an unvetted origin (management found it through a web search). Well, I would have done it if they'd given me days to study the code and days to test the integrated result. But they wanted it done in an afternoon. Fortunately, I got unemployment checks afterward, because I didn't have to quit (as I planned) and they were too embarrassed to claim the firing was for cause.
What if, as a software engineer, you move job and start work on a pre-existing project? Do you review the entire codebase? I'm curious to know if there is a point or a situation in which you accept that you cannot know the whole codebase and, if so, how is this different from accepting code that has been produced by and AI coding agent?
Hi Hugh. It's not about "knowing the codebase" - that's only ever really possible on relatively small projects, preferably that you wrote yourself. Most of the time coders *don't* know the codebase - they can be far too big and complex to "know". It's about *confidence in the code*. Say you move jobs and come onto an unfamiliar large project. Can you have confidence in it? Well if it's production code that's been running successfully for many thousands of hours, then you can certainly have confidence in it. It's been battle-tested ad nauseum. The same thing is true for the many libraries and modules we use routinely. Say I decide to create a website using the Django web framework. How many lines of code from the Django framework do I need to review? Zero! Because Django is already used successfully in tens of thousands of web sites and web applications. It's already battle tested to death. So I can just go ahead and install Django and start using it. (And in fact the same thing is true with computer languages themselves. Python is written in C, so do I need to review all that C code so I "know the codebase" of Python before I start using it? Of course not!)
Say I take up a ticket on that large, unfamiliar project I just came onto. Actually all I need to understand is the effect of the change I am
making. It's only the code that I add (and it's effects on the broader system) that I need to understand, and only at the moment when I submit it for review. Another coder will test and review it, and then assuming it passes inspection it will get added to the project. At that point I can "forget" the update, because we have confidence it works.
Of course if you were brought onto a project which was NOT running successfully in production (perhaps you were brought in for that very reason - i.e. because it wasn't performing well) then you would not have so much confidence, and you would need to go bug-hunting.
The problem is LLMs don't generate battle-tested code. In fact there's no way to know if the code they generate has ever been used before. One might actually have more confidence in a code snippet lifted off StackOverflow because at least it has a number of upvotes (note this is a joke!). There are no upvotes on an LLM's code, or any indication of what influenced the output. They don't even generate the same thing, if asked the same question twice.
I hope that makes sense but let me know any questions.
Got a good laugh out of the typo, Gary: " ... METR study reporting noting poo results."
And thanks for the last paragraph of your article. Every day I see intelligent people send me AI slop masquerading as news stories, but too many people don't check them out. A lot originate on my eternal irritant, to which I have never subscribed: Facebook. The most recent one was about Rep Massie receiving a thumb drive with all the unredacted Epstein Files on it. This is very dangerous, especially as we approach elections. Trust is the worst casualty of all.
p.s. I couldn't get into the WAPO guest link; wanted me to subscribe and blocked the article.
So glad to read this. I studied computer science in the mid-1970s. Our bible was Donald Knuth’s The Art of Computer Programming. Here’s a quote from Knuth:
Computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty.
https://www.paulgraham.com/knuth.html
I would really like to match AI algorithms with human algorithms for elegance and beauty. Not to mention the lack of bugs. There’s going to be a tsunami of system failures as all this AI code gets implemented.
Incidentally, the very first thing we learned in computer science was Turing’s proof that a machine can never debug itself.
You think I don't remember the details of Turing's halting theorem?
I'm quoting you exactly what our lecturer said back in 1974.
Yes, I was very happy to see some random person who has some experience saying exactly what my intuition tells me. I'm way out of the game, but I still have my intuitions intact.
Let's wait and see how this AI coding goes, OK? Come back to me in six months, I'll start keeping a list of AI programming disasters. You can keep a list of the successes.
Actually, the very first thing our lecturer told us was, "Computers are stupid, they will do whatever you say, however dumb."
These days, people think the machines are really smart.
Good luck with that.
> You think I don't remember the details of Turing's halting theorem?
Indeed I don't--misrememberings, misunderstandings of the material in the first place, compounding the two are very common.
> I'm quoting you exactly what our lecturer said back in 1974.
You don't have perfect memory. But if he said that, he's wrong -- the halting problem has nothing to do with a system being unable to debug itself.
Isn't it a consequence of Rice's theorem, which follows from Turing?
First, Fred didn't mention Rice's theorem, he made a claim about "Turing's proof".
Second: no. Both Turing and Rice's proofs show that no TM can determine a semantic property across *all* TMs, not that no TM can determine a semantic property of a *specific* TM, including itself--Turing didn't prove that no TM can determine whether it itself halts--there are in fact an infinity of TMs that can determine that of themselves (and an infinity of TMs that can't determine it of themselves), while there is necessarily *at least one* TM for which each of them can't prove it. It's important to get the universal quantifiers right, and many people don't.
And Rice's theorem only applies to *semantic* properties--many bugs, especially in type safe languages, are syntactic.
And again, *we are machines* (and only as computationally powerful as FSMs, not TMs) so Turing's and Rice's proofs apply to us as well. (See the Church-Turing thesis.)
Finally, the old saw about computers only doing what we tell them to is absurd in these days of machine learning and inscrutable neural networks--we've known better at least since Arthur Samuel's self-learning checker playing program that played far better checkers (even beating master-level players in 1962) than Samuel knew how to teach it.
chill your jingles jibal
What do stupid children like this think their achieving with such comments?
Blocked.
That was my experience coding with the AI tools: prompt 1 -> bug 1 -> prompt 2 to fix bug 1 -> bug 1 fixed introduced bug 2 -> prompt to fix bug 2 -> re-introduced bug 1 and didn't fix bug 2 etc. etc. You get the picture. After several hours, I felt drained and I reminded myself I know how to code CSS, HTML, JavaScript . Switched back to my own brain. I totally agree with Tom Gracey.
HTML is not code, it is markup. I guess if you can "code" so can AI, but neither of you can do software development. (CSS and JS are not robust general-purpose software development tools.) Too many "coders" have been led astray into calling themselves "developers" (not necessarily you personally).
Sigh.
Can we agree that some developers have a good experience using AI and some don't?
Oh please, Shut the Fuck Up; you're boring as hell. Are you working for OpenAI or Anthropic? Why are you shilling for them?
You too are blocked for dishonesty and general foolishness
Even before AI there was a lot of really bad code out there made by mediocre programmers. If those mediocre programmers could understand why LLM generated code is usually sub-optimal, they would have improved their code themselves.
Instead, they now have a tool to multiply their incompetent output by orders of magnitude.
Of course it is going to be personal for them.
The people who shill for billionaires and/or billion dollar tech corporations are simply mindboggling to me
Stop being such an ass. You can disagree without being rude
What do these rude obnoxious *hypocritical* virtue signaling children with their substance-free personal attacks think they are achieving? I've been on the 'net since my supervisor Charley Kline made the first remote connection, from UCLA to SRI, and never once has one of these pompous twerps caused me to change my behavior.
Blocked.
I'm a past coder, and have used AI to write code for me last year, and it did pretty well, despite trying to delete my data which I did not ask for, and in a few instances I was not fluent in two programming languages. But the time debugging it was a huge part of my effort. That reminds me, I need to track down how to stop paying a monthly fee for it. Both the nature of my job changed and the security issues have kept me from wanting to trust it for enterprise level coding tasks.
I guess one can use AI for automatic review of code; models can point out parts of your code that do not adhere to typical patterns.
This might be a bug… or a false positive. And I can deal with false positives.
Another is for researching causes of bugs before manually debugging. But if the model doesn't get it right in the first answer, it is not likely to get it right with later answers.
Both applications seem non-harmful because there is no AI slop production involved.
Agree that in constrained reviewing tasks it can add value, but the value is more likely to be incremental increases in quality (if used right) rather than "10X productivity gains" that the C-suite thinks its getting.
I used it to review some moderately complex code the other day. The latest and greatest, Opus 4.6. Read the entire 2,000K line file into a clean context with plenty to spare, about 77% free. Asked my Claude Code (with a careful, detailed, prompt) to review the code and existing test cases and create new test cases identifying bugs or untested branches, being careful not to create redundant cases.
Pros - it did identify two cases not covered, that revealed legit bugs. This was useful to me as I'd reached the point where I was too mentally tired to identify further gaps.
Cons - it spent the rest of its time generating redundant test cases either for the two cases it had already found or for the other many test cases that already existed.
Point being, it's a useful tool that I appreciated, but even on an incredibly narrow and constrained task its competence is vastly overrated, and it doesn't "reason".
Don't get me started on what it does on a small 10K multi-file repo, or a 100K larger repo, or a project in the millions of lines spread across repos.
The issue is also that this is heavily, heavily subsidized right now. If it cost like 20×, 30×, or 40× as much … I would drop it even for my rather limited use.
Development with AI (LLMs), I think, is always a very bad idea in all but select circumstances (e.g. sandboxed mockups): it generates enormous amounts of technical and cognitive debt very fast.
But more importantly, AI development is even now, despite subsidies that are ruinous for those companies, still very expensive. It rapidly burns through token since the AI has to read in a lot of files into the context for implementing a feature.
It really gave this type of development a try and ended up with a bill of $400 for a clunky, extremely buggy, and unmaintainable Frankenstein application. Imagine if that cost me $8000 instead!
Software engineers have always been like sorcerers to the non-technical, someone that produced magic and without any way for them to verify what they said was correct or not. At best, they had a technical oracle who could advise them.
Not anymore. The non-technical vibe coder has achieved sorcery without understanding what it is doing. Hell, probably a good 50% of actual coders or more really didn't have a great understanding of what they were doing, just copied and pasted, as human LLMs, hoping for the best, feeling when it seemed right, and without any ability to reason about it.
So of course they think they have conquered coding, logic, and reason, through vibes and being a word-cel. They can pretend to be shape rotaters now.
I think the use of this example is forgivable because it's not an area that he has expertise in, and the public discourse is shouting that this is a viable use case for AI. That's the "prevailing wisdom" at the moment, let's say.
Those of us with experience making software, however, know that AI is turning out to be a net negative and the discourse about that is finally catching up. The prevailing wisdom will be set straight and people won't be able to use it as an example of a viable use case.
But I don't fault Gary Marcus for this example
It’s not “the public discourse”, it’s the experience of many “with experience making software”.
“AI is turning out to be a net negative”
But this is a different subject.
“The prevailing wisdom will be set straight and people won't be able to use it as an example of a viable use case.”
Counterfactual. It already is a viable use case. It will stop being when the unsustainable LLM business goes belly up.
This is very well put in all respects. Great comment, and I agree.
Pointless useless narcissistic comment.
Could you please leave, and go to Reddit, Facebook or X, if you cannot keep a polite tone, and insist on offending people? Or maybe try discussing on HN, let's see if others let you behave like this there. Thank you.
I get what you're saying. "(AI) coding is a practical use case" should come with the caveat that AI coding is not an automated replacement for human developers because AI requires much oversight to prevent bugs and security vulnerabilities.
I don't think that Gary wants to say that AI coding is solid, he has always said that it's good to create prototypes and drafts that should be then refined by human engineers
Which somehow resembles the 'rapid prototyping' (I think it was labeled a bit differently) silver bullet of the 1990s. That failed for the same reasons that code generation at that time failed, mostly because the stuff could don't be maintained.
Sure sounds like you don't code either. Or rather, don't code with AI.
The skills of experienced coders would be more convincing if code bases weren't riddled with bugs, that QA has to be run on every code change to make sure the code doesn't break, that platforms including OSs don't need constant updating to fix bugs (and malware exploits), and periodically there are major outages often claimed to be due to a "coding error".
At the other extreme, I recall talking to a coders for a military comms system telling me that just to alter a line of code needed documenting and paperwork that was reviewed by senior people before being allowed as a change. No doubt this excessive [rocess was a CYA and means to track where erorrs occurred.
AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language. Errors will occur. GenAI will make errors too, but by not hoping to have the AI do a complete app, but by doing it piece by piece and running unit tests, boilerplate code can be created more quickly, and mistakes corrected early. Thhe idea is to keep a human in the loop and use the AI to increase the production speed, even try out different approaches, saving cognitive effort.
Humans make miskes in all sorts of endeavors. Even this economic analysis may be a mistake, so I will wait for other economists to weigh in before accepting its claims at face value. [c.f. the famous debt/GDP growth error by Rogoff that was due to a spreadsheet error!] How many news and technical articles were just wrong written by "experienced journalists and reporters?
There is also definitely an issue of lost opportunuity costs, where investments in datacenters has starved fiunding of potentially more attractive projects and businesses which has been raised by experts before.
Marcus, and others, is correct to point out where GenAI fails badly. But that shouldn't mean calling the whole enterprise a failure. Marcus has said he hopes for real AGI. I do too, but at far lower cost and resource use than we currently do. I hope LLMs are like Newcomen's atmospheric engine that was very inefficient at converting the energy in coal to work, and was obsoleted by Watt's steam engine that was far more efficient and was the main mechanical method of delivering work for manufacture and transport.
So while we may be moving along the hype cycle from the peak of inflated expectations to the trough of despondancy, I expect genAI will find its plateau of productivity. Eventually it will evolve to something like AGI, but not necessarilly to human level AGI with our foibles, but something orthogonal, like our machines that fly, but not like birds.
Give me a fresh CS graduate over an AI any day. The fresh CS graduate can learn and improve. They can explain what they are doing. They can be tasked into isolated areas with good examples to follow.
The AI coder is more like an extremely evil coder who gets hired to sabotage your code, but make it look like it isn't sabotage. They will be productive enough you stop questioning them, and then sneak in the backdoor to kill you.
Moreover the CS grad can bring entirely novel ideas and discuss them, without becoming a sycophant on the first sign of disagreement.
> AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language.
Hard disagree here.
Among the differences are that fresh graduates and experienced coders can learn, they don't need to be reprompted with the same forever, and they can reason so they don't generate probabilistic slop that is just a superficial facsimile of following instructions, e.g. the example I gave in a reply higher up the comment thread when even with a completely self-contained 2,000 file, a clean context, and explicit prompting, Opus 4.6 can't stop itself from generating redundant test cases that all test the exact same scenario.
I would be more apt to compare it to an incompetent but very experienced developer. Anyone who has worked with this type, full of arrogance and things that seem ok, vs. a new gras who cannot do as much harm, knows the difference. As the models get better, they get better at fooling reviewers with things that look right, or are too complex to reason about, so you are left with just hoping your test cases are sufficient.
> AI is not perfect, but should be thought of as a fresh CS graduate, or an experienced coder moving to a new language.
Absolutely. 100% agree. As in “fresh CS graduate” or “an experienced coder moving to a new language” are TIME SINKS, if they are on my team then I'm moving slower and together we do less than I could do alone. Did that a few times, just recently.
And that's how it works with AI, too.
That part is the same.
Now, after half-year or maybe one year, with a good CS graduate or good experienced coder moving to a new language I would get return on my investment: we start, finally, be more productive then I was alone. With AI… this part somehow, never arrives and this blog perfectly explains “why”.
P.S. But AI is pretty decent with writing one-time throw-away scripts, I'll give you that. And you can use that. But trillion dollar investments to write some throwaway scripts sounds a bit too much, if you'll ask me.
As for coding. An LLM It might be useful in boring, non-critical, repetitive tasks, and for quick prototyping. But the problems will always remain, no matter the amount of plugins or the improvements of the LLM wrappers(Agents). The LLM gives you statistical data, an average of all the code it has been trained on. Basically, it cannot ”invent” something it hasn’t seen before, it cannot learn to code from a programming language documentation without examples. The average of the code it gives you, is neither the worst or the best. And average code is filled with bugs and other issues. This will never improve due to the very nature of the transformer model LLM is built upon. If you want code, filled with bugs(behavioral, subtle bugs(hard to detect and can linger on for years before they explode) rather than naïve bugs), is non-scalable, code with bad architecture and design, LLMs will help you with that very efficiently. For code that needs serious attention, LLMs still require a senior human guard. I don’t see that can ever change with the LLM transformer model, no matter how thick and shiny the wrapper becomes.
Here’s a possibility: he read it and disagreed. But go ahead, appeal a dozen more times—perhaps sheer arrogance will win out.
With respect, great that it works out for you. Great too that it doesn't work out for others or development teams because each has *different* goals, *different* applications in different domains, and *different* workflows.
Some software developers have to architect, and develop and support software with complex features used by thousands or millions of paying customers. Other software developers are compelled to follow company guidelines (as in my own company). Yet many others work as independent consultants or they're just experimenting like retired programmers.
The term "software" is very highly nuanced. These differences must be respected. And there is a lot of crappy software out there.
What a lot of generic non sequitur handwaving ... it actually sounds like AI slop.
The issue at hand is whether Gary should stop saying that "coding is a practical use case".
Ad hominem attacks don't work on me nor others you've gone after
(" your education was wasted on you." )
If the issue at hand is whether Gary should stop saying that "coding is a practical use case." then you should have said in the first place man.
You don't know what *ad hominem* means -- there was none in my comment; it was a criticism of what you wrote.
OTOH, "nor others you've gone after" is very much *ad hominem*.
> then you should have said in the first place man.
Read the effing thread.
Muted. No, better ... blocked.
I'm not a participant in this conversation, but I want to say as a passerby your comments come off as very aggressive and disrespectful.
I'm surprised anyone is like-voting them, but gratified that Purnima's more respectful replies are being valued more highly by the community.
I have been programming for about 40 years and managing programmers for the last 25 years. For the last half year I used Claude Code every day. If you are a very disciplined software engineer that works with design documents and requirements up front and then do testing every step of the way, you now can do magic and go 10 times faster than you ever could before. For all of those in this thread that are no active programmers and haven't used Claude or Codex I can say that you are debating the taste of a dish you have never eaten. The reality on the ground is that these tools are transformative for people who already know how to engineer software properly. They are not magic wands for the undisciplined, and they are not toys for the experienced. They are power tools, and like all power tools, they reward skill and punish carelessness.
Yes. But these tools have an immense cost, the engines driving them are unsustainable, and sooner or later the chickens will come home to roost.
But are there enough users like you to pay for the massive investment?
you are welcome to comment here - for now - but please watch your tone. i got complaints on some of your recent posts from multiple people and if they continue i will have to ban you for a bit.
Noted ... but consider that my comments received numerous likes and people attacking me for my "tone" say things like "Oh please, Shut the Fuck Up; you're boring as hell. Are you working for OpenAI or Anthropic? Why are you shilling for them?" ... so they may not be the most objective or reliable sources.
What kind of a troll has time to post 29 comments on one substack article? A GenAI bot??
Agreed. I am also a skeptic on the excessive claims around LLMs. But I use it judiciously in my work (which involves development and data engineering) now because it does have practical use cases. It's not perfect, it's not effort-free, it's not a one-size-fits-all solution and it needs well structured inputs and proper review of its outputs - but the same could be said of my dishwasher, and I get plenty of use out of that.
The use case you cited of translating code between languages is particular pertinent since the best context of all for a generative model is an already solved problem where there is no ambiguity. In a similar vein, I've also used it to very quickly understand the nuances of other people's codebases, since parsing thousands of lines of unfamiliar code as a human is laborious and doesn't play to our strengths - but getting the model to explain and cite allows to me to skip the fluff and validate the logic quickly.
It has undeniable economically useful applications - it just isn't on the level they claim (since they are perversely motivated) and if it had been sold as an expensive but useful tool for developers I suspect nobody would have cared or noticed outside of tech.
And 100% there is a reckoning on the horizon for businesses that over invest in the tech and under invest in their staff during this stage of huge subsidisation by hyper scalers (the pre-enshittification stage).
Yes.
What this tells me is about what I expected. Using these tools will be helpful, but they will not be usable by novices. The current hype is that software is obsolete because users will be able to use AI to solve their own problems with AI generated code. I was deeply skeptical that these tools can be unleashed without professional supervision. All of the current selling off of SaaS companies is predicated on a myth that their products will be dumped. More likely, those products will just be more efficiently maintained and upgraded using AI tools and the clients will still pay for them.
Yes, the hype is absurd, dishonest, and dangerous.
The “current hype”? More like “decades old hype”! Remember the LAST time something like this was happening? AI was supposed to ensure that knowledge of software development would no longer be needed! Exact quite is “If the Japanese succeed in their goals, the only language you'll need to know to talk to a computer is Japanese”. The hope, back then, was that parallel computing and AI would deliver that. And people would finally, finally, finally, be able to tell computer what they need and computer would program itself. Yeah. Read the “The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to the World” book, it's not much different from Sam Altman promises. Well, the big difference is that it's 1983th year book, not 2023 year book, but who's counting? We would see more such book in the coming years.
LLMs certainly can be used by novices in writing, assuming the novices are receiving training in fact checking, source checking and all the other stuff that novices have been trained in for god knows how long. Why does everybody think that if LLMs hallucinate they are broken? It's just a trait, born of solvable with a modicum of sense and training
The question is not whether LLMs are broken or not. The question is if they could replace a software developer. Not HELP, not SPEED UP, not “marginally improve”. But REPLACE.
If that doesn't work then there are no justification for trilions of dollars spent, it's as simple as that.
P.S. for an example of how widespread and growing use of "AI" (I quote it because LLMs are not intelligent) in code development is problematic, see https://www.youtube.com/watch?v=bZJ7A1QoUEI
It seems like you assign some kind of deep moral valence to differences of experience or opinion in this area - as opposed to just stating that you have a different experience and set of observations and simply sharing those dispassionately. I'm trying to understand why you sound so angry.
Blocked.
UGH that was with model n, model n+1 that has 3% higher score on a benchmark that has test set completely contaminated will solve everything. Now just hand over 300B you damn luddites.
Since it may not be exactly clear that was sarcasm.
So AI contributed "as little as zero" to the US economy in 2025.
How do you know it's not negative? How much time has been wasted? How much money has been wasted? What could have been done with that time and money and focus and energy, if it had been spent on something useful, something constructive, instead of trying to destroy all jobs, destroy the education system, destroy people's minds and turn them into ChatGPT zombies?
How about the time spent fixing absolutely insane AI hallucinations? I've spent two weeks of a three-week editing job just trying to fix fake references generated by ChatGPT. Most of my work in 2026 has so far just been fixing AI disasters. Productive much?
I guess it depends.
If your paid contract work is to fix somebody else's AI disasters, then perhaps it has created new jobs rather than destroy them.
GDP up. Great success!
I've commented and written many times that AI is going to bring me a lot of work. The more they realise human oversight is needed, the more they will have to call on the professional editor. I've said, business is going to be good.
My paid contract is to edit academic papers, which I've been doing full-time for 12 years without a hitch. My first assignment this year has been a total AI disaster that has taken weeks longer than anticipated and messed up my schedule badly.
It's very hard to prove the negative, that a certain reference does *not* exist, especially when the authors and journal appear authentic. Editing AI product is more time-consuming than any other work I've ever done. So I am raising my prices for AI clean-up.
Good work, Fred! I bet the price tolerance for AI clean-up is high, higher than most people might think. By the time they come to you, the project is in crisis.
I expect business will be booming!
Your biggest challenge will be to keep from going completely mad in the process. I wish you strength!
*Suprised Pikachu Face*
"I think it's pretty likely the entire surface of the earth will be covered with solar panels and data centers." ~ Ilya Sutskever
This has to be one of the bleakest, most deluded takes circulating in the industry - beaten only by the pure wishful thinking these numbers lay bare and the painfully obvious shortcomings of generative AI and conventional hardware.
If your model requires a reactor, build a better model.
The brain at 25Watts looks more amazing all the time
It took 4.3 billion years to develop (and another 9.5 billion years to build the infrastructure needed for that development).
Applause!
Where did he say this? I'm interested as it is such a(nother) horribly stupid, naive thing to say. Even without fossil fuels such energy use would heat up the atmosphere faster than greenhouse gasses are doing already. Reference? I can believe he said this, but I'd like to be sure.
Hinton (being a main source for this kind of naiveté) has quite a bit to answer for.
Source: The quote is frequently attributed to Sutskever's appearance in the documentary iHuman (Nov 2019).
It can be rented for $0.98 on Amazon Prime
It seems that comment was made in the context of him expecting ASI and that an ASI would cover the world in solar panels and data centers and that it was thus important for humans to make sure ASI is benign for humans. So this is not so much a wish but a Terminator/Matrix style of expectation/fear based on the inevitability of ASI.
Gift link, as requested. https://wapo.st/4kT3oLl
added/updated, thank you!
Here's an Archive Today link for which there's no need to create an account:
https://archive.ph/LS7MM
Doesn't work. It still requires an account creation, which is no different from the Apple link.
Requiring an account doesn't mean that it doesn't work. The link is free with a non-paying account, which is how I was able to read it. (I canceled my WaPo subscription when Bezos scotched the editorial board's Presidential endorsement.)
You do understand this is classic landing page data harvesting - i.e. your email. You will then be plagued with offers and other monetizable stuff to help turn the WaPo's finances around? A [gift] link should be a no strings attached link to opens the content to read.
Non sequitur. And don't insult me by snarking about my knowledge ... I've been on the 'net for over 50 years, hold multiple network patents, and have a mention in RFC 57 from when I worked on development of the ARPANET for the UCLA Computer Science department ... there's very little that I don't know about this technology.
We should all be worshipping you, your majesty.
All good. Then please accept that I do not want to hand over my email just to read an article for good reason. As you are aware, Substack is locking down posts that one has not paid a subscription and using the "read in the app" to do the same, probably worse given the access smartphone apps have on your data.
Agreed. New York Times is at least better on that account, although their coverage on other issues maybe sucks worse. Anything to do with transgenderism in particular -- their description of the shooter in Tumbler Ridge Canada, Jesse Strang, insisted on referring to him as a "she" and "her", and gave lip service to the claim and insane idea that he [a male] was "transitioning to female":
QUOTE; New York Times: On Tuesday afternoon, Jesse Van Rootselaar, 18, grabbed two firearms from her home and, the authorities in British Columbia said, killed her mother and 11-year-old brother. Then she traveled a mile to the Tumbler Ridge Secondary School and killed five students and one educator before turning her weapon on herself.
.... Ms. Van Rootselaar was raised a boy and began transitioning at least five years ago to female, according to the police, which identified her as female. UNQUOTE
Batshit crazy to talk about "transitioning" to another sex. They might just as well have accepted a claim that he was "transitioning to a penguin."
But here's a share link that won't oblige you to sign-up there at the Times 😉🙂:
https://www.nytimes.com/2026/02/12/world/canada/tumbler-ridge-shooting-suspect-social-media.html?unlocked_article_code=1.MFA.XNHh.av0kL4oZxJZ3&smid=url-share
I have been involved in the 'responsible innovation' conversation at various levels of involvement in nanotech, synthetic biology, robotics, neurotech, quantum tech and AI, back when it meant algorithmic design and big data. The problem is 'The Economy of Promises' first coined by Physics Professor Richard Jones in 2009 in relation to nanotechnology (where I met and worked with him). The incentives from the large government research and academic funding agencies, through to venture capital and other mainstream funding only reward hyper inflated promises about changing the world, so that's what they get.
Such a prescient blog here from 2009
https://softmachines.org/?p=449
I wrote about it for the WEF blog on What we can learn from the past for new tech, in 2017.
"The way funding works is that in order to get the money, scientists and businesses have to massively exaggerate the potential benefit of their ‘ology' - ‘an end to hunger’ (GMOs) ‘electricity too cheap to meter’, (nuclear) ‘the end of disease’ (nano) - the media love it, funders get excited and the money flows.
But this “economy of promises” is just another form of fake news, with potentially damaging repercussions:
The short-term reality can’t possibly live up to the hype and the trustworthiness of the technology is tarnished. For example, the failure of the US National Cancer Institute’s 2004 aspiration to eliminate death and suffering from cancer with nanotechnology by 2015 might have left many a little disappointed.
Regulators have to start early to consider legislation around the risks and hazards of a new technology. The only place they can start is with what scientists and businesses say they will deliver. Those developing regulation in the life sciences have also found, with hindsight, that this focus is problematic. The consideration of nano-related risks and regulation driven by the hype, compounded by the thrall of the ‘ology’, was confusing and distracted from exploration of the reality of risks and hazards.
So what is the obsession with creating life-like robots, linked to the hype about ‘The Singularity’ where within 30 years robots and people will merge and we all become ‘post-human’, doing to our ability to develop robotics for useful purposes? What will be the result of the hype about the precision of CRISPR and gene editing be when, it becomes clear it is not a panacea for every genetic disease? Are we wasting time debating the ethics of science fiction when we could be discussing the not inconsiderable impact of the reality."
The real problem here they have too much money which they can spend shovelling into the idea, puffing it up, inflating the market and the influence they have got in making it look like a solution to the big problems we face which means we swallow their bullsht.
Such a shame we couldn't go the helpful practical route in the first place and focus on solving human needs and not technological progress for its own sake, which is notoriously hard to tell the fact from the fiction in every tech.
The (mythical) Singularity isn't about a "merge", it's when AI will purportedly exceed human intelligence, become capable of refining and improving itself with no limit in a positive feedback loop, with the future of the world at that point becoming completely unpredictable and opaque to us--that's why it's called a "singularity" analogous to how the physics of a black hole is like dividing by zero and thus beyond analysis. There's a lot wrong with this story, but it has nothing to do with robots and humans merging--humans become superfluous in this tale.
Yes, you are right, I didn't get that right in 2017, should have made amends this time! Thanks for pointing out.
And thank you for your intellectually honest openness to correction.
However you slice the numbers between imports and final outputs, it’s obvious that hyperscalers and neoclouds are spending massive amounts of money on data centers in the US, which does contribute to economic growth. What is much less clear (and probably untrue) is that the final generative AI product is doing anything positive for productivity or growth. IMO *that* disconnect is what’s going to cause problems in the very near future
This is exactly my reaction. Capital is being spent, right? Contractors are building things? And paying their employees a wage? Perhaps this is money being misspent, and perhaps it will end up being a net negative in the future, but by the normal method of measuring "economic growth", huge construction projects contribute in the present.
Aye, but no.
Contractors *aren't* actually building things, or at least a whole lot less than is being claimed, as much of the data centre "growth" being reported is still on paper.
One major user of data centre space, Microsoft, has been taking its foot off the gas. Perhaps they know something...
Ah, gotcha. So the news I'm reading is about pledges to build, rather than actual building?
Indeed.
By this standard, Keynes’ observation about paying people to dig holes and then fill them in again also “contributes to economic growth” in the sense that GDP goes up.
That is NOT AT ALL the same thing as productive economic activity.
Sure it is: The hyperscalers pay construction companies to build the data center shells. This means they have to pay the contractors/subcontractors, buy concrete, wood, metal, etc. They buy chips from Nvidia/AMD/etc, and the employees of those companies make money. All of those people take their earnings and go buy stuff at the store, pay their mortgage, etc. It is definitely productive economic activity!
In fact, Keynes's whole *point* about paying people to dig ditches was that even though it seemed like pointless busywork, it would be a valid method of government stimulus during a recession/depression when aggregate demand falls. And paying people to do *something* helps give people more purpose than just mailing them checks.
Two things can absolutely be true at the same time:
(1) The boom in GPUs and data centers is stimulating the economy
and
(2) The ultimate end product of that investment (I.e. generative AI/LLMs) is not useful enough to get ROI on that expense, and that's not even counting the numerous negative societal effects
I'm also bearish on GenAI and worry this bubble is going to blow up, but if we can't honestly assess the situation and throw Econ 101 out the window we're going to sound like hysterics and lose any credibility.
Well stated, thank you. It's unclear if the WaPo article claims (2) or if it claims that (1) is not actually true because that is just still on paper. Or a combination of both.
Any insight?
I skimmed the article and it seems like a very technical (pedantic?) argument that because so much of the upstream materials like GPUs are imported from foreign sources, which is subtracted from some calculations of GDP, that it's having like a net zero effect. I'm not really sure it works that way. Even for GPUs that are manufactured overseas, a lot of the money is going to US companies like Nvidia, AMD, etc.
Makes sense, thanks!
I agree with you right up until the mention of Econ 101, which desperately needs revision, absent which throwing it out the window would actually be beneficial.
BREAKING: First semester college course for beginners does not capture full nuance of complicated field, details at 11.
Seriously though, just because the basics don’t cover every conceivable corner case in a modern economy does not mean we should throw out the fundamentals. You may be shocked to learn that there are also oversimplifications in “Bio 101” and “Physics for humanities majors”
It's worse than merely "incomplete" -- it's entirely risible.
It's all overstated. It's like...AI is writing all the shit about AI...
Which really happens. In The Netherlands, a bunch of lobbyists wrote a 'national AI plan'. They (wrongly) suggested as written at the behest of the government and in it they say that they wrote it together with an LLM. Horrible piece of cheap nonsense, trying to to get money by startups and VCs and some policy wishes from ASML.
"I don’t know that Generative AI was literally a scam, but the people selling it have tried to sell it as if it were tantamount to artificial general intelligence, when it’s not."
In every iteration, electronic parrots based on large language models passed off as 'artificial intelligence' have always been a scam.
Or as I said it last August-
'AGI- A fraud wrapped in a scam tied to a bamboozle.'
"No amount of laminated computation will beget emergent properties of intelligence, let alone awareness, let alone self-awareness."
https://iandouglasrushlau.substack.com/p/agi-a-fraud-wrapped-in-a-scam-tied
To be fair, the scope of the WP article is relatively narrow and doesn’t seem to bear the weight of the conclusions being drawn from it. The article is largely silent on the alleged productivity gains, economic effects or use cases of LLMs. Rather, it focuses purely on the distribution of the value from AI investment, arguing that a substantial share of the value accrues to foreign producers rather than the US, given the significant proportion of imported computing equipment and components embodied in AI investment to date.
The AI models we have now, with their staggering, absurd costs to train them, should be seen as expressions of totalizing faith rather than a bold infrastructure project (like AI companies try to sell them).
More like Egyptian pyramids, not like Roman aqueducts.
What we gained and learned from those models, I'm sure could've been achieved much cheaper. Without getting into this frenzied race, gambling the whole economy on it, and ignoring all the massive collateral damage.
In the end, those models even lack the grandeur of the pyramids… Instead, they are embarrassing and brought out the very worst of humanity.
An excellent analogy.
I've spent the last week bouncing between things written by people whose opinion I respect on how Claude Code is amazing and game changing and things written by other people whose opinion I respect on how it generates crappily-written non-secure slop and I'm so tired right now. It did help with a little project but I'm not ready to generalize to 'rewrite Salesforce' from that.
Seems difficult to draw strong conclusions when the article specifically points out the economic data are complicated to measure, leading them to be potentially misused for any narrative:
“It’s clear that the huge spending on AI is adding to the U.S. economy, but the available economic data doesn’t neatly capture its effects. The debating economists and the slippery data suggest that if the technology does start to reshape the economy, it may be challenging to detect and clearly measure. That may leave political and corporate leaders to *choose the numbers that fit their preferred narratives* on how AI is changing American life and work.”
At least our young men and woman can play serious hockey.
When one of the biggest competitors is not allowed to play.... just saying. They beat the Canadians anyway
This was a textbook Microsoft strategy of embrace, extend, extinguish applied to an nascent industry that looked as though it could threaten their os monopoly.
They inflated openai's initial valuation with a ton of compute credits only spendable in azure. Forcing openai to do repeated mass funding rounds to keep the lights on while they use those credits.
They gassed up the media and legal realms with talk of how world changing the technology would be.
The talked of making Google dance.
They flopped on every single attempted rollout of their own ai integrations, mostly passing people off in the process. Copilot is still hot garbage.
They massively inflated nvidia purchases leading to a tragedy of the commons scenarios for big tech companies which forced them to buy under the assumption that if they were behind it would worse than if they overspend. Now they have accumulated massive debt with more to come and they are blowing through their free cash.
Microsoft hasn't been first to an industry since os and every industry they have entered since has been a wedge to protect windows not to actually compete well within that industry. They are not innovators, they are smotherers.