But I wasn’t buying, and I wrote an essay here called Peak Bubble, which read it in part:

• OpenAI doesn’t have $300 billion dollars

• They don’t have anywhere near $300 billion dollars

• By their own (presumably optimistic) projection, they won’t turn a profit until 2030.

• And all this from a company thought (or claimed) that GPT-5 was going to be tantamount to AGI (spoiler alert: it wasn’t)

• For good measure Oracle doesn’t have the chips they would need to fulfill the contracts, or even the cash to buy them.

I won’t say that it is all make-believe, but, well, you do the math…

If Oracle actually collects its $300 billion, I will truly be astounded.

Today, even after a very strong week, Oracle stands at just $170.

But my point then wasn’t that Oracle would never rise from that point (it actually did, briefly), but that in hindsight some years from now, people would think that that spike in Oracle’s stock was the peak of absurdity.

Maybe I spoke too soon.

As of today, there’s a new kid in town:

I can’t top this summary of the above, and won’t try:

Subscribe now

On the happy side, in its current form, Myth is nowhere near as scary as Tom Fridman (who worries about schoolchildren accidentally taking down power grids) and others made it out to be.

On the darker side, it really does arm attackers to a greater degree than Mythos’s predecessors.

Here’s the scariest part of the thread on X:

and here’s the part (see second paragraph) that gives a little bit of comfort:

One hopes that by now no mission-critical infrastructure is “small, weakly defended, and vulnerable” with ready network access. One hopes.

You can read a longer report here.

I agree with their conclusion that

Even if Mythos was somewhat oversold in the media, the time to get our cybersecurity house in order is now (or better yet last year) — especially given the sudden profusion of agent-written code that may in fact be both weakly defended and vulnerable.

Subscribe now

I wrote it up (quoting Josh Wolfe’s legendary tweet about how “Apple [had] just GaryMarcus’s LLM reasoning ability”) in an essay called A knockout blow for LLMs, If you have forgotten the situation you can go there for a refresher. It was (and remains) the most popular essay I have posted here.

In part I concluded

What the Apple paper shows, most fundamentally, regardless of how you define AGI, is that LLMs are no substitute for good well-specified conventional algorithms. (They also can’t play chess as well as conventional algorithms, can’t fold proteins like special-purpose neurosymbolic hybrids, can’t run databases as well as conventional databases, etc.)

…

Worse, as the latest Apple papers shows, LLMs may well work on your easy test set (like Hanoi with 4 discs) and seduce you into thinking it has built a proper, generalizable solution when it does not.

At least for the next decade, LLMs (with and without inference time “reasoning”) will continue have their uses, …. But anybody who thinks LLMs are a direct route to the sort AGI that could fundamentally transform society for the good is kidding themselves. This does not mean that the field of neural networks is dead, or that deep learning is dead. LLMs are just one form of deep learning, and maybe others — especially those that play nicer with symbols – will eventually thrive.

At the time many LLM fans were hopping mad, convinced that the Apple paper was biased and must be flawed.

Hundreds of thousands of them (maybe more) circulated a critique of the Apple paper called “The illusion of the illusion” that turned out to be AI-generated and error-ridden, literally written as joke, in the tradition of the Sokal hoax.

In reality, the replies to the Apple reasoning paper were weak sauce (see also this).

Another paper later last summer on “the mirage of reasoning” made similar arguments around chain of reasoning models, and several other papers, like the recent Stanford taxonomy of LLM reasoning errors, have since extended the point. LLMs have persistent weaknesses around planning, reasoning and generalization.

Which is a key part of why many of us have always urged for neurosymbolic hybrids, where neural networks do pattern recognition, and classical AI techniques help out around planning and reasoning.

A new paper from Tufts (released in late February, but just shared with me yesterday) picks up where the Apple reasoning paper left off, and does three interesting things.

• First it (conceptually) replicates and extends the Apple reasoning paper, by showing that a newer variant on LLMs, called VLAs –Vision-Language-Action models, which are increasingly popular in robotics — also suffer from the same generalization problems on the Tower Hanoi, similarly working fine on small versions of the problem but breaking down as more rings get added. (Humans who master Hanoi develop more robust solutions.)

• Second, they show that a neurosymbolic hybrid model, which integrates both a neural networks (for pattern recognition) and a symbolic planner generalizes far better.: “On the 3-block task, the neuro-symbolic model achieves 95% success compared to 34% for the best-performing VLA. The neuro-symbolic model also generalizes to an unseen 4-block variant (78% success), whereas both VLAs fail to complete the task”.

• Third, they show that the neurosymbolic hybrid was vastly more efficent in terms of energy use - by nearly two orders of magnitude. LLMs are an efficient way to pattern recognition where perfect results are not required, but an inefficient way to reason a plan. Different tools for different jobs. A good hybrid neurosymbolic system is about picking the right constellation of tools for a given job.

You can read the full paper by Timothy Duggan, Pierrick Lorang, Hong Lu, and Matthias Scheutz, which very much vindicates the Apple reasoning paper, here.

§

Still, as I wrote yesterday (and also six years ago, in The Next Decade in AI) taking a neurosymbolic approach is not a panacea. It’s just a starting point.

For example, the specific model here is purpose-built; we would like a general purpose system that can induce what is needed for the particular occasion; that’s not this. Claude Code (in particular version 4.6), which does seem genuinely impressive to me and a number of friends I trust, still makes plenty of mistakes, and still can’t be trusted. As one of those friend, deeply positive but still measured, put it, Claude Code is still best treated as a tool, rather than a complete solution. It’s obviously not AGI.

What we have now are lights into the future, not full answers.

But there is more and more reason to think that neurosymbolic AI is a stronger starting point, with potential for better generalization and more efficient performance.

As a society we could spend another trillion dollars pursue LLMs, or a trillion dollars seeking new, hybrid approaches. I for one know which I would pick.

Subscribe now

Claude Code, an impressive and possibly game-changing “coding agent” for programmers to write code faster is the single biggest advance in AI since the LLM.

And the thing is, Claude Code is NOT a pure LLM. And it’s not pure deep learning. Not even close.

That changes everything.

The source code leak proves it. Tucked away at its center is a 3,167 line kernel called print.ts.

print.ts is a pattern matching. And pattern matching is supposed to be the *strength* of LLMs.

But Anthropic figured out that if you really need to get your patterns right, you can’t trust a pure LLM. They are too probabilistic. And too erratic.

Instead, the way Anthropic built that kernel is straight out of classical symbolic AI. For example, it’s in large part a big IF-THEN conditional, with 486 branch points and 12 levels of nesting — all inside a deterministic, symbolic loop that the real godfathers of AI, people like John McCarthy and Marvin Minsky and Herb Simon, would have instantly recognized.1

Putting things differently, Anthropic, when push came to shove, went exactly where I have said for 25 years that the field needed to go: to Neurosymbolic AI.

That’s right, the biggest advance since the LLM is neurosymbolic. AlphaFold, AlphaEvolve, AlphaProof, and AlphaGeometry are all neurosymbolic, too; so is Code Interpreter; when you are calling code, you are asking symbolic AI to do an important part of the work.

Claude Code isn’t better because of scaling. It’s better because it is neurosymbolic. Anthropic accepted the importance of using classical AI techniques alongside neural networks — precisely the marriage I have spent my career advocating.

It’s both massive vindication for me personally (see my 2019 debate with Yoshua Bengio for context, or my 2001 book, The Algebraic Mind), and for the hundreds of other researchers who have stood by neurosymbolic AI even when prominent people like Geoff Hinton wrongly disparaged us for years.

Still, Claude Code ain’t perfect, or even close.

What we really need to do to get trustworthy AI rather than the current unpredictable “jagged” mess, is to go in the knowledge-, reasoning-, and world-model driven direction I laid out in 2020, in an article called The Next Decade in AI, in which neurosymbolic AI is just the *starting point* in a longer journey.

Read that article if you want to know what else we need to do next. The first part has already come to pass. In time, other three will, too.

Meanwhile, the implications for the allocation of capital are pretty massive: smartly adding in bits of symbolic AI can do a lot more than scaling alone, and even Anthropic has now discovered (though they haven’t acknowledged publicly) scaling is no longer the essence of innovation.

The paradigm has changed.

Subscribe now

P.s. for a good recent review of neurosymbolic AI, read this:

1. Where Tom Fridman worried in his Times column yesterday about kids accidentally blowing up the power grid…

   

   … the actual system tested was given a much easier job than in real life, with “sandboxing” turned off, making it more of proof of concept than an immediate threat.

2. Open-weight models can already do a fair amount of what Mythos can do, in a simplified preparation. Mythos is more sophisticated but perhaps not head-and-shoulders the way it was portrayed.

3. The model itself is incrementally better than previous recent models, but certainly not an off-the-chart breakthrough:

To a certain degree, I feel that we were played. The demo was definitely proof of concept that we need to get our regulatory and technical house in order, but not the immediate threat the media and public was lead to believe.

Subscribe now

Tom Friedman panicked about it in the NYT, too.

Since I haven’t gotten to play with Mythos (which has not been public released), there’s a lot I don’t know, but here are a few bets:

1. It’s probably not as bad as they say, as AI and cybersecurity expert Heidy Khlaaf explains in this thread, which I highly recommend:

   As a cybersecurity friend said to me in a text: “I haven’t had time to go through all of the write-ups yet, but it smells overhyped to me. Oh, we have this powerful model, but you can’t evaluate it yourself. I don’t doubt some of the results, but what’s made of it and the conditions under which the vulnerabilities were found, and what role humans played, isn’t clear… To me, the conditions and the scenarios are what matter. My gut tells me that if they released it publicly, we’d have some advancements but far from the exponential benefits they seem to be implying. It seems to me they are planting seeds in the hype garden.”

2. Whether or not Mythos is AGI per se is a red herring. (It probably isn’t; it’s telling that the report says very little about overall capabilities.) Crucially, AI doesn’t need to be AGI to cause harm! ChatGPT can’t reliably run a timer but it has still been implicated in delusions, suicides, cognitive surrender, mass disinformation, and so much more. Mythos may wreak havoc even if it hallucinates and lacks reliability outside of domains like coding and math. A system doesn’t have to be AGI to carry risks.

3. The strongest lesson here is about policy. Anthropic showed some admirable restraint in not publicly releasing a potentially dangerous technology.1 But some of their competitors (such as OpenAI and xAI) might well not. Whether Mythos is as scary as it sounds or not, the reality is that without any government oversight on what can be released, we are entirely at the mercy of individual CEOs, some of whom have decidedly not earned our trust.

4. Corollary, as pointed out to me by a friend who read a draft of this essay, “it is impossible to disentangle real concerns from fear mongering being used as a marketing strategy, and so it just is not possible to separate justified panic from mere advertising which is why we need government oversight!”

5. What about China? We need an international agency and treaty. That’s what I have been arguing all along. That was the point of my 2023 TED talk and 2023 invited essay with Anka Reuel in The Economist. We still do. Mythos (and the reporting around Altman) only make that clearer. Self-regulation is too little, too late.

The situation is this: three years of self-serving and misleading arguments about how regulation would allegedly preclude innovation has left us up shit’s creek, without a paddle.

I can’t tell exactly how far Mythos (or similar competitive systems that will no doubt follow) leave us up that creek, but I do know that the time for crafting paddles is running out.

Subscribe now

PS. It is worth rereading this warning I issued in January 2025, when asked by Politico for a black swan prediction:

Strong reporting in The New Yorker by Ronan Farrow and Andrew Marantz makes the case significantly more strongly.

I will let you read the details, of which there are many, for yourself, but here’s a nice summary of Altman that rings true to me:

Read it here.

§

As a postscript, another thing I have noted about Altman is that when he is in trouble, he often tries to change the narrative with hype,

Right now he is in trouble. People are increasingly concerned about the economics of his company, and in big scoop The Information just reported that the even CFO has concerns:

Today’s new OpenAI report on Superintelligence (allegedly a New Deal for the AI era) isn’t about something real or imminent (as far as I can tell), instead I suspect that it’s just another Altmanian attempt to distract, in this case from the fact that the economics of his company make no sense.

§

Now, go read that article in The New Yorker.

And think about whether we might need some outside review on this man’s decisions (as I argued in Taming Silicon Valley), or whether it would be just fine to leave our fate as a species partly up to him.

If some future OpenAI model could enables a massive bioweapon or cyberattack, would you really want Altman deciding, unilaterally, whether or not it is ok to release the model?

I sure as hell wouldn’t. And after you read the Farrow-Marantz New Yorker investigation, you probably wouldn’t either.

Subscribe now

and it went viral, declared as a victory for AI:

But there’s a lot more to be said that was only scarcely touched on in the article.

Like this

And this

And this brutal, compelling dissection on YouTube

which it itself built on and extending an earlier (May 2025) dissection from Futurism that The New York Times should have read and addressed in far more detail:

A friend of mine who has been tracking this for a while had sees Medvi as “a fraud-layer on top of also-scammy-but-possibly-less-illegal platforms”, speculating that “If there is any money there, they will be sued by all their suppliers and vendors, because I’m sure they’re in violation of every agreement in terms of compliance efforts, safe data handling, etc.” (The friend also is doubtful of the revenue reports, asking “why would this be the only thing they’re telling the truth about?”)

All in all, glorifying Medvi was not The New York Times’ finest hour.

And hardly the poster child AI boosters should be hoping for. Instead, as the YouTube video author, Voidzilla, notes, if anything Medvi is a warning sign — for how AI can be abused.

Subscribe now

First, Microsoft’s Mustafa Suleyman just tried to redefine superintelligence down from AI that is smarter than the smartest humans to something a lot more mundane: AI “models capable of delivering product value for millions of enterprises”.

If I am not mistaken, I guess makes cell phones and Microsoft Word … “superintelligent”.

§

Meanwhile, OpenAI, the company that just killed Sora and postponed “erotica” so that they could focus more, bought an eighteen-month-old podcast network called TBPN for $250 million.

A bunch of employees actually thought it was an April Fool’s:

If it’s not an April Fool’s what is it? I have only one guess:

A quarter-billion dollar desperation move: an effort to control the narrative, to distract away from the billion or so dollars they are losing each month this year, and rumors that their stock is losing favor on the secondary market.

§

At some level the fact that these two stories broke on the same day is not coincidence. With actual AGI nowhere in sight, redefinition and spin may be all that’s left.

Subscribe now

1. Advice to employers, from yours truly, today in Fortune:

“Overall impact on productivity and return on AI investment has so far been modest. Every company is investing in AI, but so far most aren’t getting huge returns.

All this could change; probably someday it will—but most likely not until we see more radical advances in AI, which could be a decade or more away. In the meantime, the advice is simple: Don’t focus on replacing humans. Focus on how you can use AI to help the ones you’ve got.”

In support of this, I give nine arguments; here are few:

You can read the rest at Fortune.

2. There is a parallel lesson for employees, of course. Don’t panic, at least not yet.

Subscribe now

This has been a war filled with mistakes. The US underestimated Iran’s resilience, overestimated the chances of regime change, and failed to anticipate Iran’s countermoves.

Yet presumably the US made use of AI in its strategy. It looks like AI helped with operations, but was not good on strategy.

Why should this be? I see at least three reasons off the top of my head, all of which should have been obvious before the bombs were dropped.

First, strategy, more than e.g., writing requisition memos, requires a broad, deep understanding of the world, but generative AI has never had robust models of the world.

Second, strategy (especially in a situation like this) demands an ability to project beyond past data, and into new situations. Generative AI has never been very good at that.

Third, there is the well-known tendency of generative AI to be sycophantic; to tell users that their ideas were the greatest ever, even when they are not. It’s not altogether difficult to imagine certain members of the senior leadership at the Department of War succumbing to that.

AI is fine for writing memos; nobody should rely on it to plan a war or to guess its likely outcomes.

Subscribe now

The examples are delicious.

To be a card carrying CEO said a thing journalist, here are some handy ground rules:

The upshot?

“The result is a sort of alternative reality journalistic simulacrum that kind of looks like journalism, but genuinely isn't interested in any context or truth that upsets the apple cart.”

You can read it in full here.

“Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided, we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. "

AGI this stuff ain’t.

This study reinforces what Anh Totti Nguyen has been saying for a long time, in a series of underappreciated papers like Vision Language Models are Blind that I keep trying to draw attention to.

Also, re the very active discussion on AI and jobs: although some white collar jobs (e.g., entry-level coder or market research assistant) may be in near-term jeopardy, many of those that require visual understanding (architect, cartographer, civil engineer, film editor, medical illustrator, urban planner, etc) probably aren’t vulnerable until entirely new techniques are developed.

And humanoid home robots? Don’t make me laugh. If your humanoid robot can’t understand the visual world, it’s just a demo, and not something you can trust.

Subscribe now

1. I will be having an ultra-timely conversation tonight on war and AI at with Katrina Manson, Bloomberg journalist and author of the excellent, could-not-be-more-topical new book Project Maven, about the history of war and AI, at Politics and Prose, Chevy Chase, Maryland.

2. At noon ET you can catch me briefly with Randi Zuckerberg on Sirius XM. We may even take call-ins, 844-942-7866.

3. At 12:20 ET, I will be talking about the death of Sora and the future of AI on BBC World.

Just in the last few years alone, cancer has taken my mother, one of my closest friends from childhood, my closest friend in my adopted hometown (she was only 51), and a beloved aunt who inspired and supported me throughout my life1, as well as several colleagues, including the AI pioneer Doug Lenat. And of course my own experience is far from unique; virtually everyone I know has lost loved ones to cancer.

AI is supposed to change all that. But so far it hasn’t.

A new essay by Emilia Javorsky, a physician scientist at the Future of Life Institute, offers a sobering statistic:

I don’t agree with everything in the new essay, but I agree with a lot of it. The part that I agree with most is its strong objection to technosolutionism, to the kind of fantasy that if we just had the right algorithm, cancer would be magically and immediately solved. (Dario Amodei’s ridiculous claims about doubling life span in the next decade are deluded on this front.)

Javorsky writes, for example, that

“Silicon Valley has repeatedly stormed into healthcare with the hubris of outsiders attempting to reinvent a system they do not fully understand, failing to learn from past mistakes. The repeated pattern suggests not mere execution challenges but systemic misunderstanding of what’s actually limiting medical progress.”

As Javorsky argues. sometimes failures in the drug development market have more to do with market forces than science. For example, Javorsky notes that “New antibiotics are inherently unprofitable, as they must be used sparingly to avoid promoting resistance, limiting revenue potential.” That’s not a science problem, it’s an incentives problem.

Or a disease may be too rare, or drug too expensive to produce. Or a looming patent expiration deadline might lead a company to abandon costly studies of a drug that actually works, simply because the economics aren’t promising enough. Javorsky gives the example of Tanespimycin, which was “difficult to produce [with] only limited time remaining before the drug’s patent expires [making] further investment in the drug difficult to justify financially." Even if we had perfect AI, it would only be part of the battle.

§

Javorsky is also quite right that there is – and always has been – a great deal of naivete:

Javorsky cautions against oversimplifying biology:

As a friend of mine likes to say, “that was never going to work.”

The reality is that biology is really really hard. Our bodies are made of trillions of interacting molecules, and each body is different. Many drugs that work for some people (probably) fail clinical trials because they work on some subset of people but we don’t understand biology well enough to know which is which.

§

The one thing I think Javorsky gets somewhat wrong is that she downplays just how important getting the right algorithm is, and how far we are from that point.

At the risk of redundancy, I will repeat a quote I shared recently from Eli Lilly CEO David Ricks in a different context that is very much on topic:

[AI is far from curing cancer and most other diseases.] If you just ask them to solve biology or chemistry questions, they’re not particularly good at it… They’re trained on the human language, not on the language of chemistry, physics, and biology.”

Over and over we discover drugs with AI that work on trials in animals but not safely enough in humans; we just don’t understand biology well enough. (This isn’t a problem specific to AI. According to one analysis, 92% of all drug trials that work with animals have toxicity problems in humans.)

Actually beating cancer is going to require systemic changes, both technical and economic, in how we develop and test drugs, as Javorsky convincingly argues, but also deeper forms of artificial intelligence that are better able to reason about chemistry, physics and biology.2 AI will also need to sweat the details a whole lot better.

The sooner we get there the better.

This essay is written in memory of my mother, who would have been 84 last week, my memory of my father, taken by cancer over decade ago, and all the other friends and colleagues and relatives that I have lost, and in memory of all those that we all have lost.

Another dramatic sign of changing times: Sam Altman, who ridiculed my 2022 critique of LLMs that argued that scaling would not bring us to AGI and that we would need architectures has just argued that

… on the research perspective, I bet there is another new architecture to find that is going to be as big of a gain as transformers [were] to LSTMs … So I would go look for where I can find a mega breakthrough [with AI’s help] …

You can watch here.

Note that in this talk Altman didn’t claim to have found such an architecture. That represents a significant retrenchment from his claim fourteen months ago that “We now know how to build AGI as it’s usually understood.”

§

Taken together with Musk’s recent admission that xAI was “not built right” and Zuckerberg’s delay of Meta’s latest model, the shift from three prominent tech CEOs in a short period of time is a strong sign that insiders are losing faith in pure scaling.

Hassabis is no longer on board either, and nor are Sutskever and LeCun. Nadella and Pichai have also hinted at skepticism around scaling.

The view of this substack since its inception has been that scaling would not lead to AGI — and it hasn’t.

§

Yet bafflingly, the powers that be are still contemplating spending trillions on data centers that are costly to our environment and that might ultimately require government bailouts.

With the case for scaling as a road to AGI steadily crumbling, it is time to reconsider. The bad bargain of AI data centers makes no sense.

Subscribe now

That hypothesis continues to go badly, as expensive experiments from two of the world’s wealthiest men have just shown.

On the hand, it’s become clear that Mark Zuckerberg’s latest model at Meta is good but not great, and not what he was hoping for.

And in the very same week, Elon Musk has conceded that for all the gigantic models xAI has built, xAI was ”not built right first time around”. Instead most of the founders are gone, and Musk has said that the company “is being rebuilt from the foundations up”.1

So much for pure scaling2

Two of the most expensive scientific experiments in history. Sooo much money down the drain.

All based on the strange and dubious-from-the-start religion of scaling-über-alles.

In my 2020 article, The Next Decade in AI, I foresaw all this, and urged the field to start focusing on world (cognitive) models and neurosymbolic AI. Now, maybe, we can finally move on to those projects?

So much time, money, and energy was lost chasing hype.

Subscribe now

Many of you may think that the Defense Department branded Anthropic a supply chain risk because they (Anthropic) refused to play ball on surveillance and fully autonomous weapons, or because OpenAI President Greg Brockman gave Trump $25 million dollers (matched by an equal contribution from Brockman’s wife), but Under Secretary of Defense/Defense Department CTO Emil Michael just gave an entirely different theory on CNBC. They are worried, or so he said, that Anthropic might “pollute” the supply chain.

A convenient summary of Michael’s argument is below; the relevant bit comes in the final minute of the Michael’s full interview, which you can find here:

Let’s break this down into parts, that I will call “marginal assumptions” and “wild leap of logic”.

Marginal assumptions

1. Claude has a soul. Why on earth would anyone say that, and under what definition? And what would qualify the Defense Department to judge such a question?

2. Claude has a “constitution” that “isn’t the US constitution”: true, but see the discussion of logic below.

3. The other day Claude was anxious. No, the other day Claude said it was anxious. It is a misapprehension of LLMs to confuse the things they say with what is true. LLMs have also, for example, been known to claim they have children (they don’t), or that they like to hang out with their friends on the weekend (also not true). They mimic things humans say, but we shouldn’t take them as veridical descriptions of anything happening on the inside.

4. “Anthropic believes that Claude has a 20% chance of being sentient right now”. No, Anthropic didn’t say this; Claude Opus 4.6 apparently did (“assign[ing] itself a 15 to 20 percent probability of being conscious under a variety of prompting conditions.”). Amodei himself refused to take those numbers seriously, when asked about them in interview.1 And, again, see point #3 above; Claude is next-word predictor, not an authority on its own internal states. It’s silly to take its estimates seriously.

The Wild Leap of Logic

From all this, some pretty dubious, and the fact that LLMs hallucinate (patently true) the Under Secretary argues that Claude is a supply chain risk that could “pollute” and poison the defense department.

Here Michael seems half-right, but entirely confused. I agree that putting hallucinating LLMs into war machines can be dangerous. But in no way is this specific to Anthropic.

Take sentience, for example. Anyone remember Blake Lemoine and his claims that Google’s LAMDA was sentient? If one takes a naive view of AI (believing that what it says reflects its actual own internal states directly), one can argue that any LLM is sentient. They have all been trained on consciousness talk, so to someone who doesn’t understand how they work they could all look sentient. But there is no more reason to think that Claude is sentient than to think that Google’s now long-obsolete system LAMDA was. If you actually believed that Claude was sentient and that sentience was de facto a supply chain risk, you would have to think that essentially every LLM is a supply chain risk. In which case the right choice is to stop using LLMs altogether, not to selectively punish Anthropic.

Essentially the same argument applies to Claude’s constitution, which amounts to a set of built-in guardrails. OpenAI has those, too! Just constructed differently. Everyone in the industry realizes that LLMs without guardrails are an uncontrollable menace. If you are worried about systems having guardrails, you shouldn’t be using LLMs, because you certainly shouldn’t be using them without guardrails. And there is no argument Anthropic’s system of guardrails is any more dangerous than anyone else’s. (It just happens to have a more evocative name.)

And the same argument I am making here also certainly applies to hallucinations. If you think it is bad to use hallucinating LLMs to autonomously choose targets (I certainly do), you shoudn’t use any of them. Again this is in no way whatsoever special to Anthropic. Either all the LLMs are supply chain risks, or none of them are.

§

Under Secretary Michael made some other points earlier in the CNBC interview that seem more reasonable; I understand why DoD might not want to work with a particular contractor. But the selective supply chain risk arguments just don’t hold water.

Subscribe now

Any coder with any chops at all knows that is one thing to write code, and another to debug it (and still another to maintain it, a year or a decade later, which is even harder)

And remember how Nathan Hamiel and I warned you in August that

Right on cue, big problems have indeed started to arrive. FT just reported that “Amazon holds engineering meeting following AI-related outages”:

§

A new study from Sun Yat-sen University and Alibaba reports similar observations in a new benchmark that focuses on long-term maintainability:

As Chris Laub summarized the study on X,

Alibaba tested 18 AI coding agents on 100 real codebases, spanning 233 days each. they failed spectacularly. [It] turns out passing tests once is easy. maintaining code for 8 months without breaking everything is where AI completely collapses.

In fairness, some of the latest systems have done better than earlier ones, but for mission critical systems, even a small number of errors can be deadly. As Amazon is discovering in the real-world.

We may well move to a regime in which AI writes most code — but for a long time to come we are going to need humans to fix the mess.

Subscribe now

As Wired reports, Anthropic’s support here against Hegseth’s unprecedented actions is broad:

I stand with them. Everyone should.