In the Harry Potter world, magic looks limitless, yet JK Rowling quietly draws a hard boundary. No matter how powerful you are, you cannot create food from nothing. Magic rearranges reality, but it never replaces effort, matter, or time.
That rule is not fantasy. It is physics dressed as folklore.
A lot of people talk about AI as if it abolishes limits. As if intelligence alone can replace labor, or lived experience. As if value can appear instantly just because something is generated on a screen.
But AI does not create substance. It recombines what already exists. The Roman aqueducts carried water brilliantly, but they could not create the spring. Every tool magnifies what you feed it. Garbage in, garbage out. Genius in, genius scaled.
Magic has limits. So does intelligence, artificial or otherwise. Respect them, or watch everything coherent collapse into noise.
i have some sympathy for this but also think it is overstated; if you take all intelligence to be recombination (which Fodor famously argued), then even what Einstein did is recombination but outside the scope of the ordinary.
Meanwhile GenAI rarely goes far beyond its training but other not yet invented tech might, and I think it is important to recognize that new techniques might be more sophisticated.
I really don't think what people do with their minds is just recombination - although recombination is part of it - and I think comparing it to LLM might be instructive about what the difference really is. I guess humans might introduce "mutation" to the recombination to follow an evolutionary metaphor, although it's maybe not random mutation like in evolution, but rather mutations that are likely to work. We do that, and calling it "creative intelligence" seems right.
Evolution is random mutation + natural selection, and the latter is not at all random. Even mutations aren't truly random, as cells have evolved various mechanisms to prevent damage. The vast majority of survivable mutations are neutral/benign at their inception, but provide mechanisms that can be deployed to protect the organism when environmental changes demand it. They are seeds upon which natural selection can build to create various capabilities. One particular useful sort of mutation is duplication--it allows the original genetic material to continue to perform its function, while evolution can "perform experiments" on the duplicate--the human immune system arose this way. Also the novel production of silk and venom in arachnids--see https://www.eurekalert.org/news-releases/1074229
> "Evolution is random mutation + natural selection, and the latter is not at all random."
Less that natural selection is not random, than that there are other processes in play:
QUOTE; Springer-Link: Visions of Evolution: Self-organization Proposes What Natural Selection Disposes ....
In the second section of the article, we show that these seven viewpoints may be collapsed into three fundamentally different ones: (1) natural selection drives evolution; (2) self-organization drives evolution; and (3) natural selection and self-organization are complementary aspects of the evolutionary process. UNQUOTE
And Stuart Kauffman: "... if ever we are to attain a final theory in biology, we will surely, surely have to understand the commingling of self-organization and selection. ...."
Another way of putting it is that AI will always be a subset of human experience. It can never be more than that. That to me means it's not physically possible to make an entity more intelligent than us.
I've seen at least one article make a very strong case that the laws of thermodynamics prohibits us from making an entity that is smarter than us. The same article said that our definition of AI is too weak.
I don't believe it's possible to pack 3.5 billion years of experience encoded in our genes into a machine. That is why I agree with the idea of AI as a recombination machine, that's it.
"I've seen at least one article make a very strong case that the laws of thermodynamics prohibits us from making an entity that is smarter than us. The same article said that our definition of AI is too weak."
Would that be the second law and its role in entropy? Can you send the article link? I'm wondering what the reasoning is that connects the "things fall apart" concept of entropy to the impossibility of AGI.
Mark Slight--not so fast--I would venture that what you say flows from the same misunderstanding that is evident (even to Einstein per quote below) in the people who are involved with AI--they haven't yet paid close attention to what actually occurs in cognition/intelligence when someone is understanding something.
Albert Einstein wrote: “Don’t pay any attention to what the scientists say to you, watch what they do.”
Also, the writer/philosopher Bernard Lonergan (mentioned in the citation below) wrote that scientists are quite good at what they do. It's when they get to extra-scientific pronouncements (namely, philosophical), that they become involved in the same philosophical confusions and misunderstandings that everyone else does and that have been around now for centuries. In my own work, I have found that not only was Lonergan correct, but those confusions, etc., are even harder to correct or to teach about because they've been around so long as to have become subconscious, quite influential, and anxiety-producing . . . and so need "unearthing," that is, if any real understanding is to occur.
In my view, then, your comment suggests that, like the others, you are standing in that same set of absences and misunderstandings.
CITATION BELOW:
Albert Einstein. Essays in Science, trans. Alan Harris (New York: Philosophical
Library,1934) 12, in Bernard J. F. Lonergan Collected Works of Bernard Lonergan: Phenomenology and Logic: The Boston College Lectures on Mathematical Logic and Existentialism. Vol. 18, ed. Philip J. McShane (Toronto: University of Toronto Press, 2001) 263.
What Einstein did was exhaustive search with creative use of the machinery and knowledge that he had available and that he built upon.
Current systems, as naive as they appear, are much better at the kind of fuzzy and thorough process of discovery than anything we had before.
They are not powerful enough, of course, and lack the world models to guide them. We will get there. It will take a lot more perspiration, data, sheer brute force, and improvements in architecture.
Of course. I did not mean to say Einstein mindlessly tried every single combination of things. That never works. I agree with your point about world models. My point is that world models are always imperfect, so it takes a lot of machinery around them.
You look at the current industry state and see waste. I see the foundation. World models are always partial, and they can be implicit. We are assembling the beast that is able to take on the world, and the parts are being built, including world models.
To Gary: There is much more "out there" in other-field academia-land and related writings that, if paid attention to (in serious fashion), would provide the wherewithal for major breakthroughs.
I hedge here and in other blogs because it takes much more than cursory coverage to get to the core of it, and blog participation, though I enjoy and appreciate reading the comments here, just doesn't cut it.
You might want to view my note here to Mark Slight re: an Einstein quote about scientists understanding what they are doing when they are understanding.
NOTE THE TIME FRAME FOR THESE RESPONSES. Though it might seem that Gary is responding to my note about the Network for Public Education, he is not. There is a three-day difference in posting times.
Yes. I write down the full idea and the skeleton of my reply first, then I ask AI to help polish it. I am really orchestrating the idea with AI. It feels similar to being a developer. No matter who writes the code(reply), you are always the final guardrail of the application(comment).
I would advise adding a final step of revision to eliminate the AI-speak. It has a strange bias toward negative parallelism, false drama, metaphors of quiet revolution, excessive adjectives, etc. As time goes on, these rhetorical tropes will become more and more tiresome to the reader; they are already becoming trite cliches. Two places for detailed descriptions of this problem are a) Wikipedia guidelines for detecting AI writing issues and b) this essay by Sam Kriss in the NYT https://www.nytimes.com/2025/12/03/magazine/chatbot-writing-style.html
Sure, there is a limit in both human and machine intelligence. That said, it is not true that AI only "recombines what exists". This is like saying that finding a nugget of gold in the mountain is simply a matter of striking with the hammer, so which would never result in value.
Value exists, but you need to find it, and mechanical means are more than adequate, if you explore the right space.
Novelty can be found, by both human and artificial means. The current algorithms are just too simple.
AI just cobbles together well-worn phrases and makes up a few of its own, while Xian said that they remain the arbiter, responsible for the final statements.
My comment is more directed to what seems to be Gary Marcus' unbridled enthusiasm for AI, just that it should be done differently to the current crop.
There are many problems with that point of view.
First, it seems that AI in its current form is a ruse to control people, or else why exactly such huge expenditure under the control of so few, while riding on the tail of social media as the feed?
Second, it seems very unlikely that this pattern of exploitation can be broken, on the contrary, it seems more likely that it will lead to greater harms.
Well, two fictional story lines thrown into one (BB and HP). One Nobel Prize winning in literature reference. JK Rowling/Bob Dylan adjust to a new era. We’re left with more norms after. I don’t know what happens.
So, to review, we wasted a global Green New Deal’s worth of cash on a tech we knew beforehand, in principle, couldn't deliver what it promised in order to get that global Green New Deal’s worth of cash while adding materially, not just via opportunity cost, to emissions in order to crash the economy which will entrench the fascism most of these idiot tech lords support and increasingly, in support thereof, program their algos to propagandize. And I didn’t mention destroying copyright, mass layoffs, or water.
Yes and we also managed to pollute and poison the Internet with more bots, spam and slop and consume more electricity, water, and bandwidth on unproven, mostly useless tech.
Not to mention mention accelerating climate disruption and wasting the money that could have been better spent on cleaning up yesterday's messes, rather than creating new ones...
We need a Molly Bloom-level mega-sentence to include all the evils that have been wrought ending in the market, and then society, saying, “No I said no I won’t No.” 😊
It's the Giant's footprint on the earth and everyone in it; you know, the one who wanted to steal, not just the golden eggs, but the goose that laid them.
My hope and expectation is that our TechLords will actually be comprehensively discredited, along with the rest of them. That seems to be where public attitudes are moving."Foaming the runway" won't be possible this time.
Doug Tarnopol: Yes, and a good number of democrats built their idea of winning the last election on what they thought (at that naive time) was clear support from the moguls in Silicon Valley. From that very public debacle (think that picture of the dinner at Trump's White House) I don't know if or when they can recover and reclaim whatever trust the Greater WE had in them in the first place. If they ever do get something worth keeping, who would trust them or what they say about it--but a dolt?
Children can learn a language by listening to a handful of people, while using as little energy as a lightbulb (do people remember lightbulbs?). AI learned language by swallowing the entire internet, while using as much energy as a fairly large city. Spot the difference.
What if the 360 billion spent on AI in 2025 by Amazon, Alphabet, Meta, and Microsoft were instead spent on K12 education ($6K/student/year), to invest in our existing, hyper-efficient biocompute infrastructure?
I, too, was fascinated/awed by LLMs when they first made their big splash a few years ago. But over time, I've come around to Gary's way of thinking about them, and totally get why they'll never work re: the promises Sam Altman and many others have made.
But I, like most people I'd guess, am not an expert on this technology. I've been playing catch-up these past few years on all things AI, again as I expect most people have. So it's taken me time to see and understand LLMs for what they actually are vs. the hype that's surrounded them.
What I keep coming back to now is, the people behind the hype must surely have known what Gary has been trying to tell us about LLMs these past few years, and *still* they promoted these things the way they did -- knowing the whole thing could (probably would?) all come crashing down. And still, they did it.
My only question now is, why? They had to have known this whole thing would eventually fail -- why?
I was chatting socially to some college friends who are now bankers at a christmas drinks. They said there was a big problem coming up in that they weren't hiring junior bankers anymore, preferring to rely on LLM output. Whether or not the LLM can actually replace the junior bankers, it's still not a great plan since with no junior hires, there will be no senior bankers in the future.
So why do you do it? I asked them. They said it cuts costs in the short term, and clients won't pay the fees for teams that are employing juniors.
And that seems like the problem all over. The LLM companies aren't trying to do what's rational, or right, or correct. They hyped it to make money, which they have and still will even when it crashes to earth, and even though they knew it would.
Terrell Johnson: The comprehensive point is that Gary is not merely sounding the alarm--he has kept on showing WHY, that is, providing reasonable evidence for his concerns. That's the longer answer to your question, but the shorter answer is probably more psychological: plain, old DENIAL where facts are washed away by wishes.
Correlation, not causation. He likes it because its creators flatter him. You should distrust it because the creators feel the need to flatter him. That demonstrates that they are untrustworthy and that their technology has flimsy or no clothes 🤭
ExplodeMeow: A huge topic. However, just on the Musk takeover, it was based purely on economic terms--applied to what was fundamentally a humanitarian project. (Oh, well.) And on hindsight, apparently the economics didn't work either, besides all our private information going to 20+-year-old technofascists who have no idea or love for democratic principles. Also, already bought stuff sat in high-rent warehouses to rot while people died. Bean-counting OR saving lives. You choose.
This is similar to Trump's recent stoppage of already-bought (big-time investments, approved by Congress) and literally sitting on the docks, windmill and energy-saving materials. (Don't get me started.)
But it might be good to look into it--many historically extant authoritarians are sociopathic morons who have (somehow?) gained massive political and social power and, I don't know you, but my guess is you won't be happy if what's going down continues. I wouldn't wait for the fascist morons to sxxt on your dinner table.
BTW, apparently a good number of Heritage Foundation people (remember the document that is meant to replace the U.S. Constitution?) have finally "gotten it" and are running away, but towards a Pence-inspired organization that, on first glance, looks something like the old, but now really renewed GOP. (See NYTimes article. If I can find it I'll post it here.)
They first create an enemy, whether it's the "far-left", "extreme DEI", even the recent "AI Race", then declare they must eliminate it, positioning themselves as the "savior".
My reason for suspicion is simple: as a Hong Konger, I recognize this tactic as common in China (especially Taiwan).
Additionally, I suspect Trump's actions predate his election, his presidency merely confirmed my suspicions (though I couldn't convince others).
Regarding the fascism concern, while I don't fully grasp the term, but I already live in a dictatorship.... the recent fire proves everything.
If this continues, China will prevail.
Every Chinese researcher or user I know (myself included) understands the nature of LLM.
I even shared a Chinese video introducing Gary Marcus earlier.
ExplodeMeow: Thank you for your informative response. It does sound like the same method. Also, create violence and be sure that everyone thinks it came from "the other side." It's nothing new; Hitler did it.
The best in show, however, is to make it so bad for your "opponent," that they finally strike out (similar to the battered wife thing) then use that provocation to excuse mass murder.
ExplodeMeow: Here is the brief NYTimes headline article about the wind energy stoppage. (The picture was amazing.)
Dec. 22, 2025, 10:11 a.m. ET/BREAKING NEWS
Brian Snyder/Reuters
Trump Halts Five Wind Farms Off the East Coast
The Interior Department said the projects posed national security risks, without providing details. The decision imperils billions of dollars of investments. Read more
In the mouths of right wingers, "DEI" is code for the n-, c-, b-, and f- words combined, plus every other form of bigotry.
P.S. Regardless of the largely incoherent and strawman drivel in the response, it doesn't contradict the fact that "DEI" in nearly every public usage in the U.S. today is a bigot's dogwhistle, and legitimate uses have been displaced, in fact forbidden by the fascists. I don't know or care much what goes on in the gamer community (I do vaguely remember gamergate), but I very much doubt that "behaviors deliberately provoking the gaming community" is a real thing.
First, regarding "DEI" specifically, as an indie game developer myself, I've observed that terms like "DEI" or "anti-DEI" are widely used within the gaming community, among both creators or players, it has used as a synonym for political censorship.
While this involves some (right-wing?) political maneuvering, it's akin to how many people treat "AI" as a synonym for GenAI or LLM.
Therefore, someone who uses the term "DEI" isn't necessarily a right-winger, but may just be an ordinary gamer, or even not an American.
(This is similar to Gary Marcus being called "anti-AI")
Personally, I use "extreme DEI" to describe behaviors deliberately provoking the gaming community; just as I insist the term GenAI, instead of using the term "AI".
Finally, I believe "extreme DEI" mirrors the "the battered wife thing" analogy mentioned earlier; I've long found it peculiar that this "CCP-style" tactic is applied to both sides.
Yes, there is quite a bit of (potential) usefulness. However, the economics of it all are extremely ‘iffy’. And we do not know yet how not to have the useful (i.e. reliable) ’signal’ drown in (unreliable, AI-slop) ‘noise’.
The ‘potential of what remains to be done’ is quite an interesting phrase from both a rational and a psychological perspective.
Anthropic is on track to break even in 2028. Google will keep on investing tens of billions to stay ahead and keep itself relevant. Overly ambitious and poorly focused players will fail.
As to what remains to be done, I don't think the architecture is set in stone. It is clear that huge amount of data, compute, and feedback from how billions of people use the tools are large enablers, which were not present in previous AI waves.
For me (not for you), the idea that these fundamental architectural breakthroughs will come is a Hail Mary in these discussions. Especially since we have been using the same basic architecture (RNNs) for more than 30 years now and only a single invention (transformers in 2016-2017) made a huge training scale-up possible (but with a costlier inference) that led to the current wave. People have been engineering the hell around the limitations of that architecture, especially by making inference bigger and bigger and finding optimisations.
For me (not for you), the idea that we have all that data (which might be exhausted more or less) and compute and feedback from billions of users and that thus these fundamental breakthroughs will come is like saying that many people have bought cauldrons and have given potions to their customers, so the philosopher's stone is just a different recipe away.
Remember 1995-2000? Many *honestly* were convinced that internet would bring perfect and free information to all people, everybody would be perfectly informed, peace and democracy would arise everywhere and the economy would be a 'long boom' (with eventually a singularity to boot) — the 'new ecoomy'. As everybody would be perfectly anonymous on the net, we would have absolute freedom of speech, and laws on information would be a thing of the past (including really bad stuff by the way). The outcome was said to be inevitable.
People are now living in a world full of manipulation and information warfare, to which unimaginable amounts of AI-slop is being added as we speak. There will come useful things of GenAI, but it will exist in a sea of slop.
Convictions (assumptions, beliefs) make our intelligence fast and efficient, they are our mental automation. And like all automation it is hard to change (somewhat easier to create). It's useful to test our convictions regularly by looking for our own doubts. Other people's doubts about one's own convictions generally have little impact.
You are confusing philosophical issues with technical ones. Nobody's saying perfect world is awaiting or inevitable.
"Especially since we have been using the same basic architecture (RNNs) for more than 30 years now"
Living things have been using the same nervous system for 400 million years, yet humans arose from that. It did not take a new architecture. It took scale and refinement.
"People have been engineering the hell around the limitations of that architecture"
It is a mistake and very simplistic to think of recent innovations this way.
By your logic, we've been engineering the hell around the steam engine since 1700.
LLM is one building block. We are building other building blocks.
A good illustration is mathematics. We have had formal verifiers for decades. What we lacked is intuition and pattern matching. Now we have that. The system is close to being complete.
Another example is AlphaGo. Brute force could not crack it. Too hard. But with a prediction engine to guess most likely moves, we did.
Human intelligence is fundamentally about pattern matching and learning from feedback. You do that enough, and you build an intuitive model of what works, that you refine by practicing. There is no more to it.
We now have all the pieces, but they are not accurate enough, not integrated enough, and not tested enough in a loop. It will take time. Likely another decade.
"Living things have been using the same nervous system for 400 million years, yet humans arose from that. It did not take a new architecture. It took scale and refinement." Mostly true. But that is based on the physical behaviour of those neurons and their environment. 'Scale makes intelligence/improvements appear' is not a law. And what works for human neurons doesn't necessarily work for digital ones (pretty unlikely, even, but that is too long a story for now). Note that it wasn't inevitable that human level intelligence would arise.
"Another example is AlphaGo. Brute force could not crack it. Too hard. But with a prediction engine to guess most likely moves, we did." AlphaGo is a narrow solution. Even more impressive is AlphaFold, also narrow, which I stated in 2022 the work deserved a Nobel Prize (which it got in 2024). I'm not a total nay-sayer. Note, AlphaGo etc. *are* forms of brute force.
"A good illustration is mathematics. We have had formal verifiers for decades. What we lacked is intuition and pattern matching. Now we have that. The system is close to being complete." How much of good abduction is feasible from GenAI is unclear, some of it, GenAI can clearly do. Here scale is a problem too. If you need to produce infinite random tries (without understanding what you are doing) to find out which ones are formally correct. So, useful, yes. Can you outcalculate math formalisms? Which areas will succumb to GenAI's type of brute force? Uncertain. Don't forget that however brilliant a tool like AlphaFold is, it creates *predictions*, and while they beat previous systems on how many of these are correct, it still doesn't mean it understands protein folding even if it does best on the CASP benchmark.
"By your logic, we've been engineering the hell around the steam engine since 1700.". Nope. The steam engine was replaced by other engines with totally different architectures (explosion, electric, jet, rocket). By my logic, you will need something different than current architecture (which is a transformer-RNN at the core), and I personally will go as far that with what we have now there is nothing that suggests we will have something like AGI within decades. Not with the currently deployed architectures. And different ones? Well which, then? Why assume they will be inevitable, within a decade, even?
Several ideas are banded around, most of these do not come with actual practical ideas on *how* to do that, such as "Adding tools that do know what they are talking about". Sure. Which? And above all: how? Neurosymbolic has seen some applications, but all narrow.
We do *not* "have all the pieces" for AGI. But we do have all the pieces for a number of useful applications (in a sea of AI-slop).
The fundamental problem of LLM alone is that it does not know what it is talking about. Adding tools that do know what they are talking about, and adding strategies for how people typically solve problems, all these are not engineering the hell out of it. These are honest building blocks, that belong in any solution. A car must have tires. Adding them is not cheating or engineering the hell out of the engine.
I work in an industry with significantly more history and stability than AI. And we know what predictions for 2028 are - educated guesses. The unreliability of predictions for 2028 is one reason power companies are hesitant to increase capacity at the rate AI model developers demand.
Yeah, those are educated guesses. But Anthropic at least has a plan and figures that show that it can keep expenses in check and revenues are increasing nicely. This in contrast with OpenAI. In the last year or so it lost contact with reality. So sure, some players are out of control.
Oleg, I'm of your persuasion, but I like to read Gary and his supporters in these comments from time to time to see how they manage to keep saying "LLMs are a failure" after every new breakthrough, e.g. NanoBanana and Alibaba's Qwen releases. Of course there will be much money lost by AI investors in the years ahead, but there will be much money made as well, and much transformation. Ask the music, entertainment, movies, and publishing industries if LLMs are ovehyped.
Those industries are being destroyed... not changed.
Humans are having to rebuild the organisations that actually facilitate and communicate human to human life experience in parallel to those "industries" that simulate that facilitiate for the expression of human phenomena.
Your comment is an example of support for the simulation spiral....leading us further away from humanity and into the unhuman experience: the simulacrum. Somebody in a body is who must be heard... the simulation is pointless and empty... and we can hear that hunger for the real even as we have gorged on the tech... to our guaranteed dissatisfaction.
Perhaps that dissatifaction is what is so appealing to the tech capitalists: the masses wanting more Big Macs... but only the real really IS. We feel it, we need it
Powerful matrices do a good job of reassembling DNA snippets, creating a cross section image from CAT opacity measurements and similar large data set solving. This is rote, computationally burdensome straight line work.
These sets generally have one and only one consistent solution, like the systems of linear equations we solved in high school but with millions of coefficients for millions of variables. Still, just one set of variable values makes it all consistent.
As any linear algebra student will tell you, such systems have one, none or an infinite number of solutions depending on how complete and consistent the equation coefficients are. LLMs just take systems that really have an infinite number of solutions and use word proximity and other iffy restriction mechanisms to shave off all but one of the solutions.
Even your GPS navigation can extrapolate multiple possible (and reasonable) routes for a second round of analysis taking travel time into consideration across the several road segments. Still it usually gives you choice. Human thought finishes the process. Thinking of LLMs this way avoids heartache.
Because they’re carnival barkers — they know that the “freak show” isn’t real, but that enough rubes will fall for the con long enough to separate them from their money … 🤨
At a guess, it's because the investors are low on capital and the AI labs are running out of excuses as to why their products are stagnant and continuing to not turn a profit. When every dollar you make costs $2, and your business is growing constantly, you eventually run out of other people's money
Why? Llms are likely pretty darn good at taking reams of otherwise largely inaccessible data and processing it to profile millions of people for control ends...
Indeed, LLMs do have some use cases. It's a tool, a limited one but in some areas they're quite useful.
The hard bit is putting a $ value on how useful they are. I still think open source local LLMs will ultimately take over. The models are already out there.
The model training "fair use" copyright cases currently in court are lose / lose for the industry. Either it's illegal, in which case they'll get fined hundreds of billions, or it's legal, in which case anyone can train their LLM on 100TB from Anna's archive or the rest of the pirate sphere. They can hack and decompile proprietary code and train their coding model on that. Scan books in, you get the idea.
So in $ terms, revenue from this technology is likely to be much lower than even the most bearish estimates.
They are trying to replace meaningful work on real problems with drudgery and gig work on past problems and hypotheticals. If the later resulted in true savings, it would work, but it is more costly, less efficient and often unreliable.
Not to mention the political backlash of raising consumers’ electric bills while telling them to just deal with job losses.
People--even those who should know better--mistake LLMs for intelligent because the models exploit the statistical patterns of language, the persuasive nature of most writing, and hope. The irony is watching tech CEOs use the exact same formula on investors—compelling narratives, statistical confidence, and an audience eager to believe.
Saw that Nadella laments all the GPUs he's got that he can't power up. Huang is also whinging about the lack of power, he says power has been "victimized". Trump's issued an EO purporting to take swift corrective action, but that's impossible "in the relevant time frame", (if at all). We're talking years to build power plants.
Couple this with scant revenue and non existent profits up against the higher interest being demanded, we shouldn't have too long to wait, now.
And the long-term viability of a Business Model that requires stealing other people's Intellectual Property and selling it as your own is questionable.
First major crack could appear in 6 days if SoftBank can't fully raise the promised 30 billion funding round for OpenAI by 1 Jan 2026. We know Sam Altman won't shut up when he gets the least little hint of success or public support. Yet he's pretty quiet on this. Maybe something else is going on behind the scenes but it is not crazy to suspect SoftBank may still be scrambling at the last minute even though they've had many months to put this package together. IF they can't get it done, than Altman's idea of a 100 billion dollar next round on a presumed valuation for OpenAI vanishes and the vultures start to circle. Add to that, we recently learned OpenAI has been propped up by 360 day payment terms from CoreWeave. Normal payment terms in tech is 30-60 days. 90 is super generous. That OpenAI gets 360 means they can underreport how much they're spending and over-report cash on hand.
A eye-opening 12/12 Atlantic article on CoreWeave. Three commodity traders became bitcoin miners. When that foundered they adopted a new name, rented out their data centers, then attracted businesses that ordered data centers to be built that they could then rent. Construction costs are from CoreWeave and its investors. Its March 2025 IPO was the year's largest. Its stock is now less than 50% its peak a few months ago. The article has more industry detail. Could it be a house of cards? CoreWeave and others purchase chips using their existing chips as collateral, but data center GPUs wear out with 2-3 years of use and depreciate in value as better chips come along.
Absolutely possible. There is a lot of writing on the length of Chip Life and depreciation. Apparently Google has gotten some thing like 7 years out of their tpus? Other estimates put the depreciation cycle on a much shorter timeline and I personally don't know enough about the nuts and bolts about chip life and depreciation to have an opinion. But if it is true that the gpus have a sharply falling value after 3 years, then. Yes the whole thing is a House of cards.
Yes, chip life unclear. I found estimates of 1-3 years with heaviest use; longer if switched to less intensive uses at some point. Did Alphabet begin testing their chips in 2018 to get a 7-year estimate? (My 2006 Prius hybrid battery lasted twice as long as predicted--they didn't yet know.)
One argument about getting more years out of existing GPUs is that the newer ones coming (like Rubin) won't have the power necessary to run them in the USA. It will be cheaper to just keep using what they have. They are also not a drop in replacement and will require a rip and replace of cooling and network. The building shell should be usable at least.
World models are not needed to play chess, do mathematics, fold proteins, and write code for some purposes, though fields such UX are there because for other purposes world models are needed. Unfortunately, fields such as AI and HCI have spent half a century (and other fields such as human factors, literature, and philosophy even longer) learning that building world models can be extraordinarily difficult. Investors in most world models will have very long waits for returns.
Playing chess very much does require a world model--AlphaZero isn't actually "zero" because it has the rules of chess hardwired in. The same is true of the others--they have a semantic framework.
P.S.
"You use the narrow one, I use the broad one"
You don't understand what words mean. I include the semantic framework of AlphaZero in the category of world model, you exclude it--therefore my definition is broader.
But this isn't what's relevant ... what is relevant is that LLMs do not have such semantic frameworks, and the thing that LLMs don't have, whatever you call it *is* needed to play chess etc.
Hi Jibal, I use "world model" to include a sense of "the world." The world that AGI would have to understand. Not just a chess board, pieces, and rules. Including a model of people.
I just asked GenAI to define world model. Its answer included this:
"People use “world model” in different scopes:
Narrow: a model of a constrained environment (e.g., chess rules)
Broad: an intuitive, human‑like understanding of the real world, including people, physics, and social dynamics
Researchers increasingly use the broader definition."
You use the narrow one, I use the broad one that it concludes is being used more.
"World model" means intuitive understanding of a field. It need not be a physics-based model or some other kind of precise simulation.
Nowadays a lot of the heavy lifting is done with neural nets. They are approximate, but can improve when given more data, and suffer less from the problem that many phenomena are too complex to be carefully coded up.
We don't need superintelligence we need wisdom. AI is pressing on the accelerator as we speed towards ecological collapse. Rather than stepping back and coordinating on how to stop amd reverse the 6th mass extinction, which will likely take us as it does all the other species, we build giant text autocomplete machines that accelerate emissions and, ultimately, collapse.
"Without world models. you cannot achieve reliability. And without reliability, profits are limited."
World models are not going to magically fix the problems.
The root of all evil, if you wish, is complexity. For this reason alone, the author wildly underestimates just how much progress we've made. We made a giant crack in the problem, and we are just getting started.
World models are being added, in piecemeal manner, where they make sense, and as our understanding grows. The reason this is not straightforward is because they need to operate flexibly and seamlessly, and putting the whole puzzle together is what is keeping us busy.
So, the future is bright, the methods are getting better and more diverse, and applications are growing too.
It's entirely possible that Marcus is too pessimistic about the capabilities of LLMs, though I think the hyperbole of their boosters (all diseases cured within 10 years! current models smarter than 50 people at once!) is much worse and has marginalized general recognition of the many limitations that remain, to no general benefit. But forgive me for not seeing a bright future even if, or perhaps especially if, those limitations are overcome. Safety and ethics have stopped being even corporate buzzwords, to say nothing of practical realities.
Among the problems that have not been solved:
- Avoiding the impoverishment and political disenfranchisement of the masses of humanity in direct proportion to how much stock and property they have. Note that earlier automation (e.g. in the 90s) already increased income inequality, and the explicit goal now is "one model to do it all."
- Avoiding incredible amounts of resource usage and pollution. Most "advancement" is coming from increased resource usage, not increased efficiency.
- Avoiding intellectual property infringement and preserving space for human creativity and self-expression.
- Avoiding risks to human safety. The current approach, somewhere between "dangerous outcomes are fine if they don't happen more than one time in a hundred," "dangerous outcomes in any quantity are fine as long as we cannot be held accountable" and "apparent behavior is a perfectly faithful reflection of long-term model behavior/’goals’," is absolutely unsafe.
- Avoiding any issues with the rights of models themselves! With "doing anything a human can do" as the explicit goal of most companies now, how will you know when or if a model or its instantiation has become a person deserving of rights? Or something that is not a person, but should still have rights? And if so, of which rights? I certainly don't know the answer to that, but I know that simply not thinking about it because it's inconvenient for profits is not it.
- Avoiding the further concentration of power in the hands of people who control the "one model to rule them all."
The worst part is that the biggest boosters don't even think about solving these problems! They just think that they'll reach some technological level, and then the solutions will just fall out of the technology, which is as naïve as it was to believe that about fossil fuel pollution 150 years and hundreds of millions of pollution-related deaths back. Remember back when people were thinking about having solutions to these problems before they created AI, or AGI, or ASI, or whatever one's favorite buzzword is? Because I do.
"containing probabilities about which propositions are true/good and which false/bad), which is Kahneman's System 2"
I think the probabilities are still system 1 though, not system 2. So LLM are in fact able to estimate probabilities very well.
System 2 is more like using the vast knowledge you have to iteratively solve a problem. You make a guess, you take a step, and see where that brings you. What you call "experimental model" is the world model, so the intuitive knowledge of how things behave.
If you don't take into account world models you get lost as most steps would be not helpful to getting you to the goal. So, I agree that System 2 is needed. I think we are getting there, as recent AI products also make use of tools and external knowledge and validation, which implicitly guide it on the right path.
This doesn't make much sense to me. Dogs and primates have no language but they have emotions and world models. "Patchwork" world models will be primitive and flawed. Please identify the progress you repeatedly talk about, I am genuinely open-minded but haven't seen it. I recommend reading Karpathy's blog. He is modest and recognizes that humans have a world model embedded in genes shaped over millions of years that envelopes emotions, cognition, language, and dreaming. He is optimistic, figures it will take 10 years, and seems unconcerned about whether investors will wait that long or what will happen if they don't.
I did not say world model have to contain language. A world model is a prediction engine. It is tuned based on experience. You can have world models produced from image data, from motor control data, etc. Then, even physics-based models will work for some things.
"humans have a world model embedded in genes shaped over a million years that envelopes emotions, cognition, language, and dreaming"
This may be true, but is not useful for implementation.
"Please identify the progress you repeatedly talk about"
Current AI systems do not just hallucinate text. They are able to create a plan of work, and go through the steps, calling tools where needed. The tools contain world models, so implementations of phenomena that are too complex to put in text.
The progress is measured by seeing how much better the AI assistants get as they make more use of tools and of reflection upon their work.
In the case of a common work process I examined, an AI system called up a logical plan of work. It was the plan of work that organizations do rely on. The problem is, in real life, as more than one person doing that work explained, "it never goes according to plan, exceptions always arise. Some arise so often we then find 'exceptions to an exception.'" I'm sure that in reality sometimes the work does follow the script, but the LLM produced the script with no mention of exception-handling, which Gary has pointed out LLMs may filter out. When canvassing for candidates in 2024, I was given scripts to follow based on quantitative research and told not to deviate. But it was obvious that on many doorsteps I should, and did. In a strongly Catholic neighborhood don't lead with freedom of choice and so on. Many years ago I phonebanked in a large room, told to follow a script. When I deviated, the manager across the room heard and came over to yell at me. I then stuck to the script but never again phonebanked. Exception-handling was a major factor (along with revenue shortfalls) in killing hundreds of thousands of task-focused chatbots 2016-2020, and the early signs are that they could do the same to millions of AI agents 2026-2030. Or maybe not. We'll see.
It is true that no plan survives the real world. That is why an AI should carefully execute the steps and do frequent validations and sanity checks. Sometimes it will get it wrong no matter what. That is why at the current stage AI is best used under very strict supervision. I believe any lessons we learn at this phase will be baked into new iterations of the products.
In the Harry Potter world, magic looks limitless, yet JK Rowling quietly draws a hard boundary. No matter how powerful you are, you cannot create food from nothing. Magic rearranges reality, but it never replaces effort, matter, or time.
That rule is not fantasy. It is physics dressed as folklore.
A lot of people talk about AI as if it abolishes limits. As if intelligence alone can replace labor, or lived experience. As if value can appear instantly just because something is generated on a screen.
But AI does not create substance. It recombines what already exists. The Roman aqueducts carried water brilliantly, but they could not create the spring. Every tool magnifies what you feed it. Garbage in, garbage out. Genius in, genius scaled.
Magic has limits. So does intelligence, artificial or otherwise. Respect them, or watch everything coherent collapse into noise.
i have some sympathy for this but also think it is overstated; if you take all intelligence to be recombination (which Fodor famously argued), then even what Einstein did is recombination but outside the scope of the ordinary.
Meanwhile GenAI rarely goes far beyond its training but other not yet invented tech might, and I think it is important to recognize that new techniques might be more sophisticated.
I really don't think what people do with their minds is just recombination - although recombination is part of it - and I think comparing it to LLM might be instructive about what the difference really is. I guess humans might introduce "mutation" to the recombination to follow an evolutionary metaphor, although it's maybe not random mutation like in evolution, but rather mutations that are likely to work. We do that, and calling it "creative intelligence" seems right.
Evolution is random mutation + natural selection, and the latter is not at all random. Even mutations aren't truly random, as cells have evolved various mechanisms to prevent damage. The vast majority of survivable mutations are neutral/benign at their inception, but provide mechanisms that can be deployed to protect the organism when environmental changes demand it. They are seeds upon which natural selection can build to create various capabilities. One particular useful sort of mutation is duplication--it allows the original genetic material to continue to perform its function, while evolution can "perform experiments" on the duplicate--the human immune system arose this way. Also the novel production of silk and venom in arachnids--see https://www.eurekalert.org/news-releases/1074229
> "Evolution is random mutation + natural selection, and the latter is not at all random."
Less that natural selection is not random, than that there are other processes in play:
QUOTE; Springer-Link: Visions of Evolution: Self-organization Proposes What Natural Selection Disposes ....
In the second section of the article, we show that these seven viewpoints may be collapsed into three fundamentally different ones: (1) natural selection drives evolution; (2) self-organization drives evolution; and (3) natural selection and self-organization are complementary aspects of the evolutionary process. UNQUOTE
https://link.springer.com/article/10.1162/biot.2008.3.1.17
And Stuart Kauffman: "... if ever we are to attain a final theory in biology, we will surely, surely have to understand the commingling of self-organization and selection. ...."
https://www.goodreads.com/quotes/4109-if-biologists-have-ignored-self-organization-it-is-not-because-self-ordering
Cutting and pasting is innovation...not true creativity.
Another way of putting it is that AI will always be a subset of human experience. It can never be more than that. That to me means it's not physically possible to make an entity more intelligent than us.
I've seen at least one article make a very strong case that the laws of thermodynamics prohibits us from making an entity that is smarter than us. The same article said that our definition of AI is too weak.
I don't believe it's possible to pack 3.5 billion years of experience encoded in our genes into a machine. That is why I agree with the idea of AI as a recombination machine, that's it.
"I've seen at least one article make a very strong case that the laws of thermodynamics prohibits us from making an entity that is smarter than us. The same article said that our definition of AI is too weak."
Would that be the second law and its role in entropy? Can you send the article link? I'm wondering what the reasoning is that connects the "things fall apart" concept of entropy to the impossibility of AGI.
Brian, I refer to this article here:
https://ui.adsabs.harvard.edu/abs/2021arXiv211107765L/abstract
I added my own perspective here:
https://scottcdunn.substack.com/p/not-so-scary-robots
This isn't to say we can't make smart machines. I just don't think anything we create can be more than a subset of our experience.
Smart machines and AGI may alway be distinct from human consciousness.
What else are you suggesting Einstein did? Adding some extra spice to the recombination, ex nihilo?
Recombination is all we have, Gary! It's all biology has.
Neither of those is true.
So what did he do then?
Mark Slight--not so fast--I would venture that what you say flows from the same misunderstanding that is evident (even to Einstein per quote below) in the people who are involved with AI--they haven't yet paid close attention to what actually occurs in cognition/intelligence when someone is understanding something.
Albert Einstein wrote: “Don’t pay any attention to what the scientists say to you, watch what they do.”
Also, the writer/philosopher Bernard Lonergan (mentioned in the citation below) wrote that scientists are quite good at what they do. It's when they get to extra-scientific pronouncements (namely, philosophical), that they become involved in the same philosophical confusions and misunderstandings that everyone else does and that have been around now for centuries. In my own work, I have found that not only was Lonergan correct, but those confusions, etc., are even harder to correct or to teach about because they've been around so long as to have become subconscious, quite influential, and anxiety-producing . . . and so need "unearthing," that is, if any real understanding is to occur.
In my view, then, your comment suggests that, like the others, you are standing in that same set of absences and misunderstandings.
CITATION BELOW:
Albert Einstein. Essays in Science, trans. Alan Harris (New York: Philosophical
Library,1934) 12, in Bernard J. F. Lonergan Collected Works of Bernard Lonergan: Phenomenology and Logic: The Boston College Lectures on Mathematical Logic and Existentialism. Vol. 18, ed. Philip J. McShane (Toronto: University of Toronto Press, 2001) 263.
What Einstein did was exhaustive search with creative use of the machinery and knowledge that he had available and that he built upon.
Current systems, as naive as they appear, are much better at the kind of fuzzy and thorough process of discovery than anything we had before.
They are not powerful enough, of course, and lack the world models to guide them. We will get there. It will take a lot more perspiration, data, sheer brute force, and improvements in architecture.
guided search not exhaustive afaik
Of course. I did not mean to say Einstein mindlessly tried every single combination of things. That never works. I agree with your point about world models. My point is that world models are always imperfect, so it takes a lot of machinery around them.
You look at the current industry state and see waste. I see the foundation. World models are always partial, and they can be implicit. We are assembling the beast that is able to take on the world, and the parts are being built, including world models.
to Oleg Alexandrov: (See also my earlier note to Mark Slight re a quote from Einstein.)
There is allot of "there" there in your note, but just one thing is to understand how and why one changes from one model to another.
Yes, Like making good use of unrestricted Negative Capability. Ideation from the ether?
To Gary: There is much more "out there" in other-field academia-land and related writings that, if paid attention to (in serious fashion), would provide the wherewithal for major breakthroughs.
I hedge here and in other blogs because it takes much more than cursory coverage to get to the core of it, and blog participation, though I enjoy and appreciate reading the comments here, just doesn't cut it.
You might want to view my note here to Mark Slight re: an Einstein quote about scientists understanding what they are doing when they are understanding.
NOTE THE TIME FRAME FOR THESE RESPONSES. Though it might seem that Gary is responding to my note about the Network for Public Education, he is not. There is a three-day difference in posting times.
“That rule is not fantasy. It is physics dressed as folklore.”
This whole comment reads like AI but this line especially
Yes. I write down the full idea and the skeleton of my reply first, then I ask AI to help polish it. I am really orchestrating the idea with AI. It feels similar to being a developer. No matter who writes the code(reply), you are always the final guardrail of the application(comment).
I would advise adding a final step of revision to eliminate the AI-speak. It has a strange bias toward negative parallelism, false drama, metaphors of quiet revolution, excessive adjectives, etc. As time goes on, these rhetorical tropes will become more and more tiresome to the reader; they are already becoming trite cliches. Two places for detailed descriptions of this problem are a) Wikipedia guidelines for detecting AI writing issues and b) this essay by Sam Kriss in the NYT https://www.nytimes.com/2025/12/03/magazine/chatbot-writing-style.html
Sure, there is a limit in both human and machine intelligence. That said, it is not true that AI only "recombines what exists". This is like saying that finding a nugget of gold in the mountain is simply a matter of striking with the hammer, so which would never result in value.
Value exists, but you need to find it, and mechanical means are more than adequate, if you explore the right space.
Novelty can be found, by both human and artificial means. The current algorithms are just too simple.
I agree with this. I don’t think Gary Marcus quite gets it.
a. People think and speak.
b. World models require models of the which come from ... see a.
You think that gibberish is getting it?
Although the answer I liked was formed by AI.
Does that invalidate it?
AI just cobbles together well-worn phrases and makes up a few of its own, while Xian said that they remain the arbiter, responsible for the final statements.
My comment is more directed to what seems to be Gary Marcus' unbridled enthusiasm for AI, just that it should be done differently to the current crop.
There are many problems with that point of view.
First, it seems that AI in its current form is a ruse to control people, or else why exactly such huge expenditure under the control of so few, while riding on the tail of social media as the feed?
Second, it seems very unlikely that this pattern of exploitation can be broken, on the contrary, it seems more likely that it will lead to greater harms.
So yes, I think that gibberish is getting in.
I was referring to this meaningless sequence of symbols:
> World models require models of the which come from ... see a.
And now you write a whole lot of stuff that is not to be found in your previous comment.
> So yes, I think that gibberish is getting in.
PROOFREAD, man.
To Jibal jibal: Looks to me like CYA.
Great analogy, Xian!
Well, two fictional story lines thrown into one (BB and HP). One Nobel Prize winning in literature reference. JK Rowling/Bob Dylan adjust to a new era. We’re left with more norms after. I don’t know what happens.
So, to review, we wasted a global Green New Deal’s worth of cash on a tech we knew beforehand, in principle, couldn't deliver what it promised in order to get that global Green New Deal’s worth of cash while adding materially, not just via opportunity cost, to emissions in order to crash the economy which will entrench the fascism most of these idiot tech lords support and increasingly, in support thereof, program their algos to propagandize. And I didn’t mention destroying copyright, mass layoffs, or water.
Slow clap!
Yes and we also managed to pollute and poison the Internet with more bots, spam and slop and consume more electricity, water, and bandwidth on unproven, mostly useless tech.
Yes slow clap of 1 hand!
Not to mention mention accelerating climate disruption and wasting the money that could have been better spent on cleaning up yesterday's messes, rather than creating new ones...
We need a Molly Bloom-level mega-sentence to include all the evils that have been wrought ending in the market, and then society, saying, “No I said no I won’t No.” 😊
It's the Giant's footprint on the earth and everyone in it; you know, the one who wanted to steal, not just the golden eggs, but the goose that laid them.
My hope and expectation is that our TechLords will actually be comprehensively discredited, along with the rest of them. That seems to be where public attitudes are moving."Foaming the runway" won't be possible this time.
It is reminiscent of Ambrose Bierce's definition of economy: Buying a barrel of whiskey you don't need for price of a cow you can't afford.
I didn’t know that line; thanks!
That sounds more like a definition of alcoholism.
But some people gained a lot.
Doug Tarnopol: Yes, and a good number of democrats built their idea of winning the last election on what they thought (at that naive time) was clear support from the moguls in Silicon Valley. From that very public debacle (think that picture of the dinner at Trump's White House) I don't know if or when they can recover and reclaim whatever trust the Greater WE had in them in the first place. If they ever do get something worth keeping, who would trust them or what they say about it--but a dolt?
Children can learn a language by listening to a handful of people, while using as little energy as a lightbulb (do people remember lightbulbs?). AI learned language by swallowing the entire internet, while using as much energy as a fairly large city. Spot the difference.
What if the 360 billion spent on AI in 2025 by Amazon, Alphabet, Meta, and Microsoft were instead spent on K12 education ($6K/student/year), to invest in our existing, hyper-efficient biocompute infrastructure?
Exactly how would that help to concentrate wealth further? /S
and yet is still shockingly terrible at it
I, too, was fascinated/awed by LLMs when they first made their big splash a few years ago. But over time, I've come around to Gary's way of thinking about them, and totally get why they'll never work re: the promises Sam Altman and many others have made.
But I, like most people I'd guess, am not an expert on this technology. I've been playing catch-up these past few years on all things AI, again as I expect most people have. So it's taken me time to see and understand LLMs for what they actually are vs. the hype that's surrounded them.
What I keep coming back to now is, the people behind the hype must surely have known what Gary has been trying to tell us about LLMs these past few years, and *still* they promoted these things the way they did -- knowing the whole thing could (probably would?) all come crashing down. And still, they did it.
My only question now is, why? They had to have known this whole thing would eventually fail -- why?
I was chatting socially to some college friends who are now bankers at a christmas drinks. They said there was a big problem coming up in that they weren't hiring junior bankers anymore, preferring to rely on LLM output. Whether or not the LLM can actually replace the junior bankers, it's still not a great plan since with no junior hires, there will be no senior bankers in the future.
So why do you do it? I asked them. They said it cuts costs in the short term, and clients won't pay the fees for teams that are employing juniors.
And that seems like the problem all over. The LLM companies aren't trying to do what's rational, or right, or correct. They hyped it to make money, which they have and still will even when it crashes to earth, and even though they knew it would.
Machine learning as personal mentorship to inspire profiles in courage. Makes perfect sense.
Terrell Johnson: The comprehensive point is that Gary is not merely sounding the alarm--he has kept on showing WHY, that is, providing reasonable evidence for his concerns. That's the longer answer to your question, but the shorter answer is probably more psychological: plain, old DENIAL where facts are washed away by wishes.
Addendum: The first clue to steer away for someone who doesn't understand AI is that Trump likes it.
Correlation, not causation. He likes it because its creators flatter him. You should distrust it because the creators feel the need to flatter him. That demonstrates that they are untrustworthy and that their technology has flimsy or no clothes 🤭
This also is the same person who chooses cabinet heads by the same playbook as a Hollywood casting call. Also, they probably paid him.
And suggested injecting bleach to treat covid And no MAGA he wasn't kidding
Hmm... I used to be a staunch supporter of Trump.
Unfortunately, his supporters think:
"If people who hate Trump hate it, then it must be a good thing!"
So I've been working to explain the underlying purpose of tech companies; I usually use the UK's cybersecurity laws as an example.
But the most I dare say is “Trump got duped, he doesn't understand new things.”
By the way, what really made me suspicious was Elon Musk.
Ever since he claimed buying Twitter was to "fight-DEI", I stopped trusting him.
(Even I don't care for extreme "DEI", but it's clearly political posturing.)
ExplodeMeow: A huge topic. However, just on the Musk takeover, it was based purely on economic terms--applied to what was fundamentally a humanitarian project. (Oh, well.) And on hindsight, apparently the economics didn't work either, besides all our private information going to 20+-year-old technofascists who have no idea or love for democratic principles. Also, already bought stuff sat in high-rent warehouses to rot while people died. Bean-counting OR saving lives. You choose.
This is similar to Trump's recent stoppage of already-bought (big-time investments, approved by Congress) and literally sitting on the docks, windmill and energy-saving materials. (Don't get me started.)
But it might be good to look into it--many historically extant authoritarians are sociopathic morons who have (somehow?) gained massive political and social power and, I don't know you, but my guess is you won't be happy if what's going down continues. I wouldn't wait for the fascist morons to sxxt on your dinner table.
BTW, apparently a good number of Heritage Foundation people (remember the document that is meant to replace the U.S. Constitution?) have finally "gotten it" and are running away, but towards a Pence-inspired organization that, on first glance, looks something like the old, but now really renewed GOP. (See NYTimes article. If I can find it I'll post it here.)
First, thank you for your response.
My view is this:
They first create an enemy, whether it's the "far-left", "extreme DEI", even the recent "AI Race", then declare they must eliminate it, positioning themselves as the "savior".
My reason for suspicion is simple: as a Hong Konger, I recognize this tactic as common in China (especially Taiwan).
Additionally, I suspect Trump's actions predate his election, his presidency merely confirmed my suspicions (though I couldn't convince others).
Regarding the fascism concern, while I don't fully grasp the term, but I already live in a dictatorship.... the recent fire proves everything.
If this continues, China will prevail.
Every Chinese researcher or user I know (myself included) understands the nature of LLM.
I even shared a Chinese video introducing Gary Marcus earlier.
ExplodeMeow: Thank you for your informative response. It does sound like the same method. Also, create violence and be sure that everyone thinks it came from "the other side." It's nothing new; Hitler did it.
The best in show, however, is to make it so bad for your "opponent," that they finally strike out (similar to the battered wife thing) then use that provocation to excuse mass murder.
ExplodeMeow: Here is the brief NYTimes headline article about the wind energy stoppage. (The picture was amazing.)
Dec. 22, 2025, 10:11 a.m. ET/BREAKING NEWS
Brian Snyder/Reuters
Trump Halts Five Wind Farms Off the East Coast
The Interior Department said the projects posed national security risks, without providing details. The decision imperils billions of dollars of investments. Read more
In the mouths of right wingers, "DEI" is code for the n-, c-, b-, and f- words combined, plus every other form of bigotry.
P.S. Regardless of the largely incoherent and strawman drivel in the response, it doesn't contradict the fact that "DEI" in nearly every public usage in the U.S. today is a bigot's dogwhistle, and legitimate uses have been displaced, in fact forbidden by the fascists. I don't know or care much what goes on in the gamer community (I do vaguely remember gamergate), but I very much doubt that "behaviors deliberately provoking the gaming community" is a real thing.
First, regarding "DEI" specifically, as an indie game developer myself, I've observed that terms like "DEI" or "anti-DEI" are widely used within the gaming community, among both creators or players, it has used as a synonym for political censorship.
While this involves some (right-wing?) political maneuvering, it's akin to how many people treat "AI" as a synonym for GenAI or LLM.
Therefore, someone who uses the term "DEI" isn't necessarily a right-winger, but may just be an ordinary gamer, or even not an American.
(This is similar to Gary Marcus being called "anti-AI")
Personally, I use "extreme DEI" to describe behaviors deliberately provoking the gaming community; just as I insist the term GenAI, instead of using the term "AI".
Finally, I believe "extreme DEI" mirrors the "the battered wife thing" analogy mentioned earlier; I've long found it peculiar that this "CCP-style" tactic is applied to both sides.
I hope this means I am going to stop receiving invites to webinars intended to teach me how to build llms ans 'agentic ai'
Hanlon’s Razor (at least for many of the hypesters): Never attribute to malice what can be explained by stupidity.
You vastly underestimate the usefulness of what we have and the potential of what still remains to be done.
Yes, there is quite a bit of (potential) usefulness. However, the economics of it all are extremely ‘iffy’. And we do not know yet how not to have the useful (i.e. reliable) ’signal’ drown in (unreliable, AI-slop) ‘noise’.
The ‘potential of what remains to be done’ is quite an interesting phrase from both a rational and a psychological perspective.
Anthropic is on track to break even in 2028. Google will keep on investing tens of billions to stay ahead and keep itself relevant. Overly ambitious and poorly focused players will fail.
As to what remains to be done, I don't think the architecture is set in stone. It is clear that huge amount of data, compute, and feedback from how billions of people use the tools are large enablers, which were not present in previous AI waves.
Most people assume their convictions come from their observations and reasonings. In practice, the reverse is more important: our observations and reasonings are extremely influenced by their convictions (this is an evolutionary necessity, and our self-delusion may be too — https://ea.rna.nl/2022/10/24/on-the-psychology-of-architecture-and-the-architecture-of-psychology/).
For me (not for you), the idea that these fundamental architectural breakthroughs will come is a Hail Mary in these discussions. Especially since we have been using the same basic architecture (RNNs) for more than 30 years now and only a single invention (transformers in 2016-2017) made a huge training scale-up possible (but with a costlier inference) that led to the current wave. People have been engineering the hell around the limitations of that architecture, especially by making inference bigger and bigger and finding optimisations.
For me (not for you), the idea that we have all that data (which might be exhausted more or less) and compute and feedback from billions of users and that thus these fundamental breakthroughs will come is like saying that many people have bought cauldrons and have given potions to their customers, so the philosopher's stone is just a different recipe away.
Remember 1995-2000? Many *honestly* were convinced that internet would bring perfect and free information to all people, everybody would be perfectly informed, peace and democracy would arise everywhere and the economy would be a 'long boom' (with eventually a singularity to boot) — the 'new ecoomy'. As everybody would be perfectly anonymous on the net, we would have absolute freedom of speech, and laws on information would be a thing of the past (including really bad stuff by the way). The outcome was said to be inevitable.
People are now living in a world full of manipulation and information warfare, to which unimaginable amounts of AI-slop is being added as we speak. There will come useful things of GenAI, but it will exist in a sea of slop.
Convictions (assumptions, beliefs) make our intelligence fast and efficient, they are our mental automation. And like all automation it is hard to change (somewhat easier to create). It's useful to test our convictions regularly by looking for our own doubts. Other people's doubts about one's own convictions generally have little impact.
You are confusing philosophical issues with technical ones. Nobody's saying perfect world is awaiting or inevitable.
"Especially since we have been using the same basic architecture (RNNs) for more than 30 years now"
Living things have been using the same nervous system for 400 million years, yet humans arose from that. It did not take a new architecture. It took scale and refinement.
"People have been engineering the hell around the limitations of that architecture"
It is a mistake and very simplistic to think of recent innovations this way.
By your logic, we've been engineering the hell around the steam engine since 1700.
LLM is one building block. We are building other building blocks.
A good illustration is mathematics. We have had formal verifiers for decades. What we lacked is intuition and pattern matching. Now we have that. The system is close to being complete.
Another example is AlphaGo. Brute force could not crack it. Too hard. But with a prediction engine to guess most likely moves, we did.
Human intelligence is fundamentally about pattern matching and learning from feedback. You do that enough, and you build an intuitive model of what works, that you refine by practicing. There is no more to it.
We now have all the pieces, but they are not accurate enough, not integrated enough, and not tested enough in a loop. It will take time. Likely another decade.
"Living things have been using the same nervous system for 400 million years, yet humans arose from that. It did not take a new architecture. It took scale and refinement." Mostly true. But that is based on the physical behaviour of those neurons and their environment. 'Scale makes intelligence/improvements appear' is not a law. And what works for human neurons doesn't necessarily work for digital ones (pretty unlikely, even, but that is too long a story for now). Note that it wasn't inevitable that human level intelligence would arise.
"Another example is AlphaGo. Brute force could not crack it. Too hard. But with a prediction engine to guess most likely moves, we did." AlphaGo is a narrow solution. Even more impressive is AlphaFold, also narrow, which I stated in 2022 the work deserved a Nobel Prize (which it got in 2024). I'm not a total nay-sayer. Note, AlphaGo etc. *are* forms of brute force.
"A good illustration is mathematics. We have had formal verifiers for decades. What we lacked is intuition and pattern matching. Now we have that. The system is close to being complete." How much of good abduction is feasible from GenAI is unclear, some of it, GenAI can clearly do. Here scale is a problem too. If you need to produce infinite random tries (without understanding what you are doing) to find out which ones are formally correct. So, useful, yes. Can you outcalculate math formalisms? Which areas will succumb to GenAI's type of brute force? Uncertain. Don't forget that however brilliant a tool like AlphaFold is, it creates *predictions*, and while they beat previous systems on how many of these are correct, it still doesn't mean it understands protein folding even if it does best on the CASP benchmark.
"By your logic, we've been engineering the hell around the steam engine since 1700.". Nope. The steam engine was replaced by other engines with totally different architectures (explosion, electric, jet, rocket). By my logic, you will need something different than current architecture (which is a transformer-RNN at the core), and I personally will go as far that with what we have now there is nothing that suggests we will have something like AGI within decades. Not with the currently deployed architectures. And different ones? Well which, then? Why assume they will be inevitable, within a decade, even?
Several ideas are banded around, most of these do not come with actual practical ideas on *how* to do that, such as "Adding tools that do know what they are talking about". Sure. Which? And above all: how? Neurosymbolic has seen some applications, but all narrow.
We do *not* "have all the pieces" for AGI. But we do have all the pieces for a number of useful applications (in a sea of AI-slop).
The fundamental problem of LLM alone is that it does not know what it is talking about. Adding tools that do know what they are talking about, and adding strategies for how people typically solve problems, all these are not engineering the hell out of it. These are honest building blocks, that belong in any solution. A car must have tires. Adding them is not cheating or engineering the hell out of the engine.
I work in an industry with significantly more history and stability than AI. And we know what predictions for 2028 are - educated guesses. The unreliability of predictions for 2028 is one reason power companies are hesitant to increase capacity at the rate AI model developers demand.
Yeah, those are educated guesses. But Anthropic at least has a plan and figures that show that it can keep expenses in check and revenues are increasing nicely. This in contrast with OpenAI. In the last year or so it lost contact with reality. So sure, some players are out of control.
Oleg, I'm of your persuasion, but I like to read Gary and his supporters in these comments from time to time to see how they manage to keep saying "LLMs are a failure" after every new breakthrough, e.g. NanoBanana and Alibaba's Qwen releases. Of course there will be much money lost by AI investors in the years ahead, but there will be much money made as well, and much transformation. Ask the music, entertainment, movies, and publishing industries if LLMs are ovehyped.
Those industries are being destroyed... not changed.
Humans are having to rebuild the organisations that actually facilitate and communicate human to human life experience in parallel to those "industries" that simulate that facilitiate for the expression of human phenomena.
Your comment is an example of support for the simulation spiral....leading us further away from humanity and into the unhuman experience: the simulacrum. Somebody in a body is who must be heard... the simulation is pointless and empty... and we can hear that hunger for the real even as we have gorged on the tech... to our guaranteed dissatisfaction.
Perhaps that dissatifaction is what is so appealing to the tech capitalists: the masses wanting more Big Macs... but only the real really IS. We feel it, we need it
You mean because the output of the creative industries in already so bad in 2025, LLMs can hardly make it any worse ?
Powerful matrices do a good job of reassembling DNA snippets, creating a cross section image from CAT opacity measurements and similar large data set solving. This is rote, computationally burdensome straight line work.
These sets generally have one and only one consistent solution, like the systems of linear equations we solved in high school but with millions of coefficients for millions of variables. Still, just one set of variable values makes it all consistent.
As any linear algebra student will tell you, such systems have one, none or an infinite number of solutions depending on how complete and consistent the equation coefficients are. LLMs just take systems that really have an infinite number of solutions and use word proximity and other iffy restriction mechanisms to shave off all but one of the solutions.
Even your GPS navigation can extrapolate multiple possible (and reasonable) routes for a second round of analysis taking travel time into consideration across the several road segments. Still it usually gives you choice. Human thought finishes the process. Thinking of LLMs this way avoids heartache.
Because they’re carnival barkers — they know that the “freak show” isn’t real, but that enough rubes will fall for the con long enough to separate them from their money … 🤨
Shortsightedness, Greed, power, hubris, vanity, self delusion to name a few.
When there are trillions of dollars flowing around it's easy enough to siphon off a few million or even billion for yourself.
At a guess, it's because the investors are low on capital and the AI labs are running out of excuses as to why their products are stagnant and continuing to not turn a profit. When every dollar you make costs $2, and your business is growing constantly, you eventually run out of other people's money
Why? Llms are likely pretty darn good at taking reams of otherwise largely inaccessible data and processing it to profile millions of people for control ends...
It's an LLM investment bubble, calling it the AI bubble is a misnomer.
With the correct nomenclature, the bubble would likely have never existed in the first place.
The name is part of the con.
The Pythons invented "fraudulin"... AI is a fantasy...a delusion for those techies of clay.
Indeed, LLMs do have some use cases. It's a tool, a limited one but in some areas they're quite useful.
The hard bit is putting a $ value on how useful they are. I still think open source local LLMs will ultimately take over. The models are already out there.
The model training "fair use" copyright cases currently in court are lose / lose for the industry. Either it's illegal, in which case they'll get fined hundreds of billions, or it's legal, in which case anyone can train their LLM on 100TB from Anna's archive or the rest of the pirate sphere. They can hack and decompile proprietary code and train their coding model on that. Scan books in, you get the idea.
So in $ terms, revenue from this technology is likely to be much lower than even the most bearish estimates.
They are trying to replace meaningful work on real problems with drudgery and gig work on past problems and hypotheticals. If the later resulted in true savings, it would work, but it is more costly, less efficient and often unreliable.
Not to mention the political backlash of raising consumers’ electric bills while telling them to just deal with job losses.
Coupled with analysis like this, by a former IMF chief economist, the downside is indeed looking up.
Gita Gopinath on the crash that could torch $35trn of wealth
https://www.economist.com/by-invitation/2025/10/15/gita-gopinath-on-the-crash-that-could-torch-35trn-of-wealth
From The Economist
I’m no highfalutin’ financier, but $35tn sounds like real money.
A trillion here, a trillion there, and before you know it you're talking about real money! Lol
Considering it's more than the cash in circulation and the demand deposits in savings, yeah, it's a lot.
That's $100,000 for every man, woman and child in the USA
People--even those who should know better--mistake LLMs for intelligent because the models exploit the statistical patterns of language, the persuasive nature of most writing, and hope. The irony is watching tech CEOs use the exact same formula on investors—compelling narratives, statistical confidence, and an audience eager to believe.
Saw that Nadella laments all the GPUs he's got that he can't power up. Huang is also whinging about the lack of power, he says power has been "victimized". Trump's issued an EO purporting to take swift corrective action, but that's impossible "in the relevant time frame", (if at all). We're talking years to build power plants.
Couple this with scant revenue and non existent profits up against the higher interest being demanded, we shouldn't have too long to wait, now.
$1.5 T. My God.
And the long-term viability of a Business Model that requires stealing other people's Intellectual Property and selling it as your own is questionable.
First major crack could appear in 6 days if SoftBank can't fully raise the promised 30 billion funding round for OpenAI by 1 Jan 2026. We know Sam Altman won't shut up when he gets the least little hint of success or public support. Yet he's pretty quiet on this. Maybe something else is going on behind the scenes but it is not crazy to suspect SoftBank may still be scrambling at the last minute even though they've had many months to put this package together. IF they can't get it done, than Altman's idea of a 100 billion dollar next round on a presumed valuation for OpenAI vanishes and the vultures start to circle. Add to that, we recently learned OpenAI has been propped up by 360 day payment terms from CoreWeave. Normal payment terms in tech is 30-60 days. 90 is super generous. That OpenAI gets 360 means they can underreport how much they're spending and over-report cash on hand.
A eye-opening 12/12 Atlantic article on CoreWeave. Three commodity traders became bitcoin miners. When that foundered they adopted a new name, rented out their data centers, then attracted businesses that ordered data centers to be built that they could then rent. Construction costs are from CoreWeave and its investors. Its March 2025 IPO was the year's largest. Its stock is now less than 50% its peak a few months ago. The article has more industry detail. Could it be a house of cards? CoreWeave and others purchase chips using their existing chips as collateral, but data center GPUs wear out with 2-3 years of use and depreciate in value as better chips come along.
Absolutely possible. There is a lot of writing on the length of Chip Life and depreciation. Apparently Google has gotten some thing like 7 years out of their tpus? Other estimates put the depreciation cycle on a much shorter timeline and I personally don't know enough about the nuts and bolts about chip life and depreciation to have an opinion. But if it is true that the gpus have a sharply falling value after 3 years, then. Yes the whole thing is a House of cards.
Yes, chip life unclear. I found estimates of 1-3 years with heaviest use; longer if switched to less intensive uses at some point. Did Alphabet begin testing their chips in 2018 to get a 7-year estimate? (My 2006 Prius hybrid battery lasted twice as long as predicted--they didn't yet know.)
One argument about getting more years out of existing GPUs is that the newer ones coming (like Rubin) won't have the power necessary to run them in the USA. It will be cheaper to just keep using what they have. They are also not a drop in replacement and will require a rip and replace of cooling and network. The building shell should be usable at least.
Let's hope none of it involves threatening orphans with guns.
World models are not needed to play chess, do mathematics, fold proteins, and write code for some purposes, though fields such UX are there because for other purposes world models are needed. Unfortunately, fields such as AI and HCI have spent half a century (and other fields such as human factors, literature, and philosophy even longer) learning that building world models can be extraordinarily difficult. Investors in most world models will have very long waits for returns.
Playing chess very much does require a world model--AlphaZero isn't actually "zero" because it has the rules of chess hardwired in. The same is true of the others--they have a semantic framework.
P.S.
"You use the narrow one, I use the broad one"
You don't understand what words mean. I include the semantic framework of AlphaZero in the category of world model, you exclude it--therefore my definition is broader.
But this isn't what's relevant ... what is relevant is that LLMs do not have such semantic frameworks, and the thing that LLMs don't have, whatever you call it *is* needed to play chess etc.
Hi Jibal, I use "world model" to include a sense of "the world." The world that AGI would have to understand. Not just a chess board, pieces, and rules. Including a model of people.
I just asked GenAI to define world model. Its answer included this:
"People use “world model” in different scopes:
Narrow: a model of a constrained environment (e.g., chess rules)
Broad: an intuitive, human‑like understanding of the real world, including people, physics, and social dynamics
Researchers increasingly use the broader definition."
You use the narrow one, I use the broad one that it concludes is being used more.
"World model" means intuitive understanding of a field. It need not be a physics-based model or some other kind of precise simulation.
Nowadays a lot of the heavy lifting is done with neural nets. They are approximate, but can improve when given more data, and suffer less from the problem that many phenomena are too complex to be carefully coded up.
Good riddance. Stochastic parrots are not good for the community.
We don't need superintelligence we need wisdom. AI is pressing on the accelerator as we speed towards ecological collapse. Rather than stepping back and coordinating on how to stop amd reverse the 6th mass extinction, which will likely take us as it does all the other species, we build giant text autocomplete machines that accelerate emissions and, ultimately, collapse.
"Without world models. you cannot achieve reliability. And without reliability, profits are limited."
World models are not going to magically fix the problems.
The root of all evil, if you wish, is complexity. For this reason alone, the author wildly underestimates just how much progress we've made. We made a giant crack in the problem, and we are just getting started.
World models are being added, in piecemeal manner, where they make sense, and as our understanding grows. The reason this is not straightforward is because they need to operate flexibly and seamlessly, and putting the whole puzzle together is what is keeping us busy.
So, the future is bright, the methods are getting better and more diverse, and applications are growing too.
It's entirely possible that Marcus is too pessimistic about the capabilities of LLMs, though I think the hyperbole of their boosters (all diseases cured within 10 years! current models smarter than 50 people at once!) is much worse and has marginalized general recognition of the many limitations that remain, to no general benefit. But forgive me for not seeing a bright future even if, or perhaps especially if, those limitations are overcome. Safety and ethics have stopped being even corporate buzzwords, to say nothing of practical realities.
Among the problems that have not been solved:
- Avoiding the impoverishment and political disenfranchisement of the masses of humanity in direct proportion to how much stock and property they have. Note that earlier automation (e.g. in the 90s) already increased income inequality, and the explicit goal now is "one model to do it all."
- Avoiding incredible amounts of resource usage and pollution. Most "advancement" is coming from increased resource usage, not increased efficiency.
- Avoiding intellectual property infringement and preserving space for human creativity and self-expression.
- Avoiding risks to human safety. The current approach, somewhere between "dangerous outcomes are fine if they don't happen more than one time in a hundred," "dangerous outcomes in any quantity are fine as long as we cannot be held accountable" and "apparent behavior is a perfectly faithful reflection of long-term model behavior/’goals’," is absolutely unsafe.
- Avoiding any issues with the rights of models themselves! With "doing anything a human can do" as the explicit goal of most companies now, how will you know when or if a model or its instantiation has become a person deserving of rights? Or something that is not a person, but should still have rights? And if so, of which rights? I certainly don't know the answer to that, but I know that simply not thinking about it because it's inconvenient for profits is not it.
- Avoiding the further concentration of power in the hands of people who control the "one model to rule them all."
The worst part is that the biggest boosters don't even think about solving these problems! They just think that they'll reach some technological level, and then the solutions will just fall out of the technology, which is as naïve as it was to believe that about fossil fuel pollution 150 years and hundreds of millions of pollution-related deaths back. Remember back when people were thinking about having solutions to these problems before they created AI, or AGI, or ASI, or whatever one's favorite buzzword is? Because I do.
"containing probabilities about which propositions are true/good and which false/bad), which is Kahneman's System 2"
I think the probabilities are still system 1 though, not system 2. So LLM are in fact able to estimate probabilities very well.
System 2 is more like using the vast knowledge you have to iteratively solve a problem. You make a guess, you take a step, and see where that brings you. What you call "experimental model" is the world model, so the intuitive knowledge of how things behave.
If you don't take into account world models you get lost as most steps would be not helpful to getting you to the goal. So, I agree that System 2 is needed. I think we are getting there, as recent AI products also make use of tools and external knowledge and validation, which implicitly guide it on the right path.
This doesn't make much sense to me. Dogs and primates have no language but they have emotions and world models. "Patchwork" world models will be primitive and flawed. Please identify the progress you repeatedly talk about, I am genuinely open-minded but haven't seen it. I recommend reading Karpathy's blog. He is modest and recognizes that humans have a world model embedded in genes shaped over millions of years that envelopes emotions, cognition, language, and dreaming. He is optimistic, figures it will take 10 years, and seems unconcerned about whether investors will wait that long or what will happen if they don't.
I did not say world model have to contain language. A world model is a prediction engine. It is tuned based on experience. You can have world models produced from image data, from motor control data, etc. Then, even physics-based models will work for some things.
"humans have a world model embedded in genes shaped over a million years that envelopes emotions, cognition, language, and dreaming"
This may be true, but is not useful for implementation.
"Please identify the progress you repeatedly talk about"
Current AI systems do not just hallucinate text. They are able to create a plan of work, and go through the steps, calling tools where needed. The tools contain world models, so implementations of phenomena that are too complex to put in text.
The progress is measured by seeing how much better the AI assistants get as they make more use of tools and of reflection upon their work.
In the case of a common work process I examined, an AI system called up a logical plan of work. It was the plan of work that organizations do rely on. The problem is, in real life, as more than one person doing that work explained, "it never goes according to plan, exceptions always arise. Some arise so often we then find 'exceptions to an exception.'" I'm sure that in reality sometimes the work does follow the script, but the LLM produced the script with no mention of exception-handling, which Gary has pointed out LLMs may filter out. When canvassing for candidates in 2024, I was given scripts to follow based on quantitative research and told not to deviate. But it was obvious that on many doorsteps I should, and did. In a strongly Catholic neighborhood don't lead with freedom of choice and so on. Many years ago I phonebanked in a large room, told to follow a script. When I deviated, the manager across the room heard and came over to yell at me. I then stuck to the script but never again phonebanked. Exception-handling was a major factor (along with revenue shortfalls) in killing hundreds of thousands of task-focused chatbots 2016-2020, and the early signs are that they could do the same to millions of AI agents 2026-2030. Or maybe not. We'll see.
It is true that no plan survives the real world. That is why an AI should carefully execute the steps and do frequent validations and sanity checks. Sometimes it will get it wrong no matter what. That is why at the current stage AI is best used under very strict supervision. I believe any lessons we learn at this phase will be baked into new iterations of the products.