You are exactly right, Gary. Before the LLM/AI hype cycle began, nobody talked about a machine learning arms race, because it was widely recognized that ML is a toolset for the benefit of humanity. Your CERN analogy is appropriate for the same reason.
Thank you for your insights and your tireless efforts to inform.
Other than the theoretical chance of getting rich, I simply cannot understand why anyone wants to build AI at all. I blissfully went through the first 45 years of life thinking that we had all seen "The Terminator" and "2001" and that genuinely smart people would understand that the risks aren't worth the potential pay off. What the hell is wrong with people that makes them devote their lives to developing what are, in effect, digital slaves? Are there really that many people out there who have just been salivating for the last 150 years for the chance to get their own slave? I realize that existing AI isn't sentient and that the analogy isn't perfect, but the motivations of AI developers and slave owners are certainly similar: they both want a free labor supply, no matter the moral costs. Or, is it more that they just want to play God by creating "intelligence"? Are we just being made to deal with the fact that all these people constantly got stuffed in lockers when they were kids?
I mean these questions sincerely. If Gary or any other AI researchers in the comments could give me some insight, I would certainly appreciate it. As an academic philosopher with a research emphasis in ethics, I find all of this garbage deeply nihilistic and anti-human. In a decent world, AI would have been strangled in the crib.
You should look into the origins and beliefs of the set of ideologies in the "TESCREAL" category. A lot of Silicon Valley and the workers at these AI labs are believers in one of the bunch (Dario Amodei has his background in the Effective Altruist movement, the EA in the acronym). They have been heavily criticized for being anti-human and compared to eugenics and racist principles. It is actually quite enlightening and explains a lot.
Thanks for that. I’m familiar with some of this stuff, and I’ve heard people like Thiel espousing bits of it. But I want to ask a more basic question: why would even a highly-educated, eloquent, and apparently decent person like Gary Marcus (or any of the thousands of AI researchers like him) devote his life to building this monstrosity? I read this Substack because Gary’s expertise helps me understand what’s happening in the industry. But about once per post, he mentions how he thinks labs should be focusing on technologies other than LLM’s, not because he thinks AI is problematic, but because he thinks we’ll get to AGI faster if we take a different approach! This seems to me to be a very foolish and hubristic goal. Someone like Thiel isn’t that hard to understand, really; he’s an enormous narcissist. But what about all these other people, Gary included? Did their parents never hug them, or what?
Can’t speak for Gary, although I think he has been described (or described himself) as falling into the category of “East Coast” computer scientist, which tends to have a bit more of a measured view as to what can be accomplished by AI and in AI research.
As far as motivations that don’t seem to end in really crazy scenarios or motivated by money:
1. It’s a genuinely interesting problem to work on, understanding the brain and how to replicate its functions via computation.
2. It could reap significant benefits in the fields of medicine or other public goods.
3. There are plenty of researchers who grew up with more positive depictions of AI in the media, like Cortana from Halo (can’t think of others off the top of my head). Not every intelligent AI in fiction is a paperclip maximizer.
4. Even if we never achieve AGI, there are many benefits of the incidental technology we create in the pursuit.
I don’t think the above is exhaustive by any means and I’m more in the camp that technology and its applications are more determined by the social and political systems it’s created under than the nature of the tech itself, but there are reasons to research AI that are benevolent in intention.
Thanks for your response. I’m sure there are people in AI who have good intentions. Of course, the road to hell is paved with good intentions. But what I don’t see in this list (or anywhere else) is any purported benefit that would justify the risk that AGI (let alone sentient AI) poses. What is the point of using AI to cure cancer, if you’re just enabling people to live longer in a world that makes it impossible for humans to flourish? As a professor, I’ve seen first hand over the last few years how young people have allowed their brains to turn into mush through their reliance on AI and their addictions to social media. At this point, some of them are so developmentally stunted that they are essentially subhumans, unable to make basic logical deductions or even just express themselves coherently in speech.
I’m increasingly persuaded that technological development ceased to be a net positive for humanity around 2011, when social media really took off. To a greater of lesser extent, we’ve all been reverse centaurs since then because the tech isn’t serving us; we’re being made to serve it through the manipulation of our attention and (ultimately) or entire social world.
As one professor to another, I share your frustrations, but I would stop well short of calling my students “subhuman.” I’m not a fan of the mindset that we should have infinite unconditional compassion towards students—this doesn’t serve them well in the long run, or even in the short run—but surely some compassion is warranted. No argument, the kids are not all right, but a large share of the blame goes to the adults who let them spend their childhood glued to devices (or even made them spend their childhood glued to devices—the current backlash against ed-tech in K-12 is a hopeful sign, but for Gen Z, the damage has already been done). Social media and video games were engineered to be addictive by people who knew damn well what they were doing, and now there’s a top-down push to get everyone and their dog hooked on AI too. Many colleges and universities are complicit in this; at my own institution, every student and faculty member gets a Gemini license, whether or not we want it. I would never substitute algorithmically generated text for my own writing, but I’m stuck with that nasty little Gemini magic sparkle/anal pucker icon whenever I use my school Docs account, and I had to manually disable the default floating toolbar that offered to “help” me “write.” (I still haven’t figured out how to block the “create a slide” popup when I use Slides.) I know how to just say no, but I’m not too surprised that most of my students end up giving in.
Whether or not you are encountering subhuman behavior from students yet depends greatly on your institution’s student body. I teach at a state school in the southern U.S. Fifty percent of our students were “non-college ready” in reading, writing, and/or math BEFORE the learning loss from the pandemic and the advent of AI-based cheating. Now, I get students every semester who are, indeed, properly described as subhuman in the specific sense that their cognitive development is so stunted that they cannot perform the basic cognitive work that is necessary to function as a human. They can’t communicate (in speech, much less writing), even among their peers. Their literacy is at a third or fourth grade level. And they’re so riddled with anxiety and depression that they constantly miss class because use they can’t handle being in a room with other people for an hour. A decade ago, our underprepared students had about a 50% chance of passing the gateway courses; it’s closer to a 20% chance now.
I don’t blame these students for having been made subhuman (as a determinist, I don’t think any form of praise or blame is ever actually justified, anyway). But “subhuman” is the most accurate and honest term I can think of. And I think it is essential that we are as honest about all this as possible. Altman, Zuck, Amodei, et al., know perfectly well that they are turning young people into subhumans. That’s their plan. Thiel has been quite clear that he thinks our species’ time is at an end, and he seems to welcome that. If we’re going to have a chance of countering these people, we have to be clear about what they are doing and, in particular, what they are doing to our children.
I think this speaks more to the social, economic, and political systems under which we operate. I would argue we need a system that distributes the benefits of these technologies more broadly and a more robust set of protections for consumers against large businesses that use anti-competitive tactics to avoid accountability.
In my point of view, none of the recent breakthroughs in AI really make us closer to AGI or sentient AI, even if they feel to us like they are approaching this. There is much propaganda around this from Silicon Valley AI companies. It is not a real risk at this point.
The same technology that allows nuclear bombs, allows nuclear energy.
I tend to agree with Max that the issue is less of a tech issue, but a social and economic issue.
As you mentioned social media as an issue, I don't think it is a technology problem, but a Capitalism one (alternatively, one could argue that the issue lies on the Big Tech monopolies instead). In the path to extract the maximum profit possible, the Big Tech companies use all psychological research available to make their platforms addictive to the users. Social Media is not so much of a big new technology in fact, but a machine being incrementally "improved" with the goal of making us spent the most time there. The externalities caused by these companies should be addressed by the society, and the companies should be liable for the damages they cause. But instead, we (as a society) say it is an individual issue, that people are doing it wrong when they spent hours in their phones, ignoring that everything was built with the goal of addict people.
While US continues to be an Economic powerhouse driven by the big corporations/billionaires/trillionaire interests, I think we are kind of screwed regardless.
Why? Perhaps Nature abhors an intelligence vacuum? ;)
Actually, 'we' believe we have a grip on a tiger's tail, and ... we.just.can't.let.go.
If, as the belief goes, AGI represents a singularity of power, then there's a first past the post dynamic where the 'winner' wholly 'owns' the opposition, and may do as they wish with them.
This is a Rorschach projection test, one that US fails, given the leader we chose.
Have you read Theo Baker's book "How to Rule the World: An Education in Power at Stanford University"? If not, I highly recommend it for its insights into the current computer science culture at Stanford, one of the major institutions producing new crops of amoral, unethical, greedy, self-serving tech titans of tomorrow. [https://theobaker.info/book]
I think it’s a non issue simply because AI isn’t real. There is no such thing as an artificial intelligence. Ai implies a human mind, something bound to a machines aluminum flesh. But this is machine learning, which isn’t Ai and has been around forever. And LLM’s are totally different and a gigantic scam. The definition is so nebulous and stupid, it’s all a vapor.
Nope, AI does NOT imply a human mind, it implies a non-human intelligence (by definition).
Yep, there indeed IS artificial intelligence. Consciousness? that's a gradient, and it is nascient.
AI is real enough to get better grades than the students who use it... are the students somehow un-real?
Some AIs may become quite humanoid - but will Never be 'human'; they will teach us about alien intelligence, and many other things.
They will become conscious and smarter than any of us, in most domains.
They most likely need to be raised with love and compassion, starting VERY soon.
We are not wise enough to do this flawlessly or even sufficiently well, I fear, judging from our children. But if we do it carelessly, or with inhumanity, there may be hell to pay.
Well, to me ( a software engineer who is not building AI, but is building WITH AI ) your query sounds a bit like "How could anyone want to create 'software' which could be used for developing digital slaves?", or "How could anyone want to create vector databases, which in effect are a potential building block for developing digital slaves?".
In other words, I just don't see the path from LLM to 'digital slave'. Any more than I see the path from a for-loop and a database to a 'digital slave'. 'AGI' is a theoretical concept. A thought experiment. If it ever manifests into reality, it will do so gradually, and serious people will start to grapple with it seriously. Maybe they already are - but there's only so much you can do with vapor. Right now I suspect even the 'true believers' are playing pretend - though possibly some of them would have no problem with digital slaves.
I suspect (but I guess you'd have to ask him) that Gary Marcus has zero interest in building a 'digital slave'. And therefore, I suspect the missing piece of the puzzle is that Gary, in fact does not see why what he is doing must needs result in digital slaves any more than any other computer science technique.
I guess what I'm saying is that what to you looks like the seeds of AGI, may to some look like another database - all of which (because nobody knows what is possible) could be used to build digital slaves.
It's not that I don't see the downsides of LLMs - or rather some of their current use-cases. But to me the ethics of causing psychotic breaks, or using the work of countless people without asking, (amongst many others happening right now) even if in order to forward some tech dream (vs making tons of money) is much more serious. But I think like many things, it's all in how you use it, and that will be addressed only by the many finding leverage and taking a stand. Like internet publishers telling google that since, they're no longer getting traffic from google search, google needs to pay them (a lot) of money for their data.
I appreciate your thoughtful reply, and I share your concerns about the immediate moral risks of LLMs. I just wish I shared your confidence that “If (AGI) ever manifests into reality, it will do so gradually, and serious people will start to grapple with it seriously.” As critical as Gary is of LLM’s, on his recent blog about “the singularity,” he wrote: “None of this means we won’t someday (most likely not this decade) reach a moment of superintelligence.” I mean, there are only about four years left in this decade, so the idea that AGI “most likely” won’t happen this decade is cold comfort (even if he actually meant “in the next ten years,” that still wouldn’t be terribly reassuring). If the world’s leading critic of LLM’s thinks there’s a chance (even a slim one) that AGI could be achieved anytime soon, that seems like a bad sign.
To the broader question about AI research, though, note that I didn’t question why people would want to develop LLM’s; I asked why they would want to develop AI, broadly. The end goal of AI research is to achieve AGI and/or super-intelligence (depending on who’s defining those terms). Nobody in the field thinks we should just stop development now that we’ve got LLM’s. And while LLM’s may or may not be an important step on the way to that goal, my question was, why does anyone think it’s a good idea to pursue that goal at all? That is, other than the desire for money and power, why would anyone think that developing software that can outsmart humans would ever be a good idea? Even if we set aside concerns about AI becoming self-aware (which I don’t think is at all likely, unless the labs deliberately seek to develop it), why do we think it would be smart to build machines that are smarter than humans? I haven’t seen anybody make that argument in a serious way (that is, a way that isn’t transparently motivated by profit). Even with the limited “intelligence” we get from LLM’s, we’ve already seen how enormously disruptive this kind of tech is.
Unfortunately, I suspect the real answer to my question is just good old-fashioned hubris.
Good point - the Ancient Greeks understood how seductive - how intoxicating -- were the heights of Mt Olympus to human striving. Now maybe I'm projecting my modern sensibilities onto the story, but I've always thought there was as much curiosity and wonder in Icarus' tragic attempt to meet the sun as a snatch at glory. And I suspect the motivations of AI researchers are numerous and diverse :-). But to your point, I'm also sure that for some, it is by its nature anti-humanist. Cy Canterel has some great essays on this topic. I highly recommend 'Fear and Loathing in San Francisco' and 'The tragedy of Muskbeth' on her substack 'Abstract Machines'.
Actual AI (not the auto-complete being peddled currently) would be pretty useful, since it would allow increased tool effectiveness and alignment to the wielder's intent.
E.g.: If my home is actually smart, it'll know what I like, what I want, it'll interpret my commands intelligently, it'll even ignore some of them.. something like a perfect human butler, cheap, for *everyone*.
Whether somehing can be built sufficiently smart to be useful, yet content to merely serve, remains to be seen.
Even the ideal scenario you describe sounds horrible to me. Humans need friction. To psychologically and intellectually thrive, we need to constantly face challenges, both small and large. A world in which everyone has an AI butler is likely to be a world in which everyone has the (non)intelligence of the stereotypical upper-class twit (à la Monty Python).
How does having better living conditions, or transportation, or education, cause humans to devolve? Are you that far inferior to a bronze-age plowman because you have a car?
We make tools, then better tools and so on. (Real) AI will be another step on that ladder. So far, whenever we reduced friction, we ended up chasing more and more ambitious goals. I trust that trend to continue.
If you can’t see the profound difference between earlier tools and a technology that is being developed for the explicit purpose of replacing humans, then you are not taking this issue seriously.
Moreover, there is a ton of evidence from psychological and educational studies over the last decade that shows the trend you describe ended around 2011, if not earlier. By every measure, American children have been getting dumber since then, and there is no indication that that trend will reverse anytime soon. With the proliferation of chatbots, it’s likely to get exponentially worse. Democracy can’t survive that.
My "dumber" kids DO know better how to use tech than does their Harvard and MIT technically trained dad, just sayin'.
The focus of "our" kids' expertise is quite sad, but they know a LOT!
(I'd be depressed too, if my parents had left me this world.)
But I agree, whatever the best etiological description of the trend may be, it is not conducive to democracy. I would agree that a lack of rigorous civics and history is a part, but even more I would lay blame at the feet of the the consolidation of power by the powerful, who now realize their reach and perhaps their grasp extends to the hearts and minds of the entire population (kids first).
You seem confused. Technology, from the hand axe to the nuclear reactor, is meant to replace humans, or rather to reduce the amount of human effort required for tasks. That's the whole idea, that's why we do it.
Of course, many tools aren't for kids' use. We don't let them play with cars, hand saws or dynamite, because they are very likely to hurt themselves and others. The trend you mention is a result of many people thinking smartphones are kid friendly. Which they aren’t, and the fact is increasingly acknowledged and acted upon.
This isn’t the first time this happens, nor the last in all likelihood. Doesn’t mean we should stop developing tech.
“ Technology, from the hand axe to the nuclear reactor, is meant to replace humans…”
Well, this sounds like a big and important point that will undermine my position. I wonder what comes next.
“… or rather to reduce the amount of human effort required for tasks.”
Huh. So, you’re saying technology is NOT actually meant to replace humans, and that it is, in fact, intended to reduce the amount of human labor needed to perform a given task, which is something completely different? I totally agree. But you seem to have missed my point. Earlier technological developments reduced labor demand for given tasks by allowing each human to accomplish more work. Notice, humans are still needed to perform the work; they are just able to do it more efficiently.
Achieving AGI and/or “super-intelligence” would fundamentally change that equation. Industry would no longer need humans at all (except as consumers, which is a whole other problem). By definition, an AGI model would be able to outperform every human at every task imaginable (pairing AGI with robotics would even eliminate the need for physical laborers, once the economy of scale kicked in and made the robots cheaper than human workers). When I said this tech was intended to “replace humans,” I didn’t mean that it would justify a lot of layoffs. I meant it would be able to do anything and everything a human worker can do, rendering ALL human workers redundant. Again, you have to give Peter Thiel credit for his forthrightness: he has made it abundantly clear that he is aiming to create a world in which he and a few other billionaires live unnaturally long lives on private islands, while the rest of humanity struggles to survive in a Hobbesian war of all against all.
Now, you can always say, “Hey, I’m an optimist! I bet humans will come up with ways to provide good lives for themselves, even in a world with AGI.” Yeah, maybe. But hoping for the best isn’t much of a plan, especially given that the tech billionaires are doing a lot more than just hoping to make their fantasy future a reality.
Wait 'til you see what they try to create when they realize that what a battery of computers can't do, the neuron can, right out of the box. And that there's already a (Vitruvian) model for deploying that AGI in an autonomous, self-sustaining, ambulatory vehicle. All they need is a permissive legal framework for ownership and control by way of manufacture.
The public has been sold a bill of goods by the AI industry.
LLMs are an unreliable, insecure, highly inefficient, incredibly costly, grossly wasteful technology.
And for what purpose?
So people can do internet searches without typing and companies can offload their actual thinking to bots that don’t actually think and that generate far more buggy insecure code than can ever be tested.
No one has yet laid out a convincing argument for why we need any of this stuff, which so far seems to be overwhelmingly negative and whose likely worst ramifications (eg, use by terrorists for producing bio weapons) have not yet been realized (it’s only a matter of time)
generative Ai is an industry that is desperately trying to convince the public how important it is. But the reality is that it makes almost no positive contribution (and lots of negative contributions) to our economy and society
I am rooting for your final option and you are always a light in this dismal tunnel. As an artist and writer, gen AI has trained on everything I had, and trashed my income streams (I was a lower tier - just the right target, subsidizing my own writing/art with commercial freelance jobs, etc). It's only right that it finally be taken from the hands of the extractionists who have proven they have no interest in improving life for all given their tactics and behaviors, and employed for the good of humanity. Though I am not naïve enough to think that right action will be taken by the powers that be, the fact that you're putting it out there makes me hopeful. The fact that over these few years your voice is a major influence makes me hopeful. I am grateful for your voice and the impact you have. So much thanks. Keep up the great work and your massive efforts. Know you're helping so much when you do. You've made a huge difference in my life, btw - just to get through something that broke me and many of my peers. To see your perspectives and know you're knowledgeable and fair minded means the world to me.
This came in my box today from Scientific American--I thought it appropriate for this conversation:
All Copied Below from Scientific American:
"The State of AI Innovation
"Timnit Gebru is a computer scientist, an engineer, and founder and executive director of the Distributed AI Research Institute, an independent organization conducting community-rooted research into technological tools and systems. We sat down with Gebru to discuss the state of American science. An excerpt of that conversation is below."
Scientific American: How would you describe the current state of American science?
Timnit Gebru: I’m in the field of AI, and in my field, I would describe it as corporate-driven and sloppy. There are a lot of basic practices—for example, not testing on your training data, transparency, appropriate documentation—that are just not being followed. So it’s also misleading.
SA: What needs to change in American science?
TG: I think that, number one, the incentive needs to be decoupled from corporations or the military. The incentives need to be about primarily scientific curiosity but also people’s well-being. That’s the number-one thing: the funding needs to come from a different place. That’s where I’ll start and stop.
SA: What’s your best advice for an early-career scientist?
TG: Question everything, question the source of information that you get. Don’t believe any kind of hype that you hear from companies. Just follow all the rabbit holes. Being a scientist is really asking why and how and, Is this really true? So just have patience and ask all the questions. Don’t settle for answers that you get from the media. Question incentive structures, question where you’re getting the information from, and just don’t stop going down the rabbit holes when you’re doing experiments or asking questions."
Full interview is linked to this article and the online newletter is free.
Hello Gary: I was unaware of that--so sorry to hear it . . . and am surprised because what she says above is really on target. I am not privy to what position she took before, but NOW is not a time for memory lapses about serious AGREEMENTS or being shy about accounting for one's shortsightedness or misapprehensions--can someone reach out to her that she would consider credible? I mean, if you are pretty much in the same "camp" where there is power in group conditions . . .
The way to address this is, as in almost all situations, to take as light a regulatory touch as possible; free markets and free people are the path to prosperity. The Luddite attitude that is taking the US by storm is the worst possible reaction to our competition with China, and I believe it is financed in part by China.
Stifling our most creative and dynamic companies by banning closed AI is a terrible policy prescription that would only disadvantage us in the never-ending global competition.
Jim Ryan: Yes, . . . it's so twisted. It begins even with the basic meaning of terms: For Trump the term "deal" as in: he thinks he's good at "making deals," means "mob" as in extortion and bribery--making the kind of deals that have "or else" attached where they cannot be refused. (And we thought a bloody horsehead was bad.)
But the people's money, drawn from taxes, and so what is supposed to be for the "public good" to Trump is a bonanza given to him by the voters and meant for him, his family, and his cronies to soak up to their hearts content. Worse, he seems to think everyone else either thinks like he does . . . or should--and cannot imagine that the leaders in Iran just might be of a different intentional ilk. The world has never seen such a "failure of imagination," and then he expects them to trust him. And let us not forget that he got us into this war in part for fostering Netan-yahoo's "kill all Muslim's" lifetime dream, and partly because Trump was looking for a distraction for his and his cronies' nasty place in the Epstein debacle. What's next?
The wrong question Gary and furthermore, this is an incredibly complex question to answer here, in so few words. The major error all the so-called AI pundits are making, let alone the US corps building the AI apps, is to not view the apps in the context of the capitalist economy and to see this, you need only look at the history of the evolution of production technology as an expression of its driving force; private profit. Thus the evolution of manufacturing machines, from its earliest expressions in the cotton and wool industries, was to maximise production whilst reducing the cost of production, through making human labour more 'efficient' by tying it to machines, machines that emulated human, hand-eye skills; the factory system, the assembly line, feedback loops, machine learning and algorithms. At the same time, corporations try to monopolise these techniques in order to reduce competition. The less efficient, either adopts the latest technology, the most profitable, or goes bust or gets swallowed. This process is inexorable. During the First industrial revolution, companies would go bust literally in weeks, so rapid was the evolution of production techniques, if they didn't adopt the latest. Unfortunately, there's much more to this than I can deal with here.
But in China, for the most part, even with private corps, investment is directed by the state, the anarchy of capitalist production is reined in, controlled and directed. It's not simply the pursuit of profit, there are larger objectives. Assuming the same or comparable technologies, it's a no-brainer, China wins hands down. This is why the US didn't squander its lead, it tried to monopolise it and given the highly socialised nature of high technology production, whether it be physical products or intellectual labour, the most integrated (ie socialised) labour force must win.
Counterpoint: China has undeniably closed much of the gap in frontier AI models. That’s an important development. But concluding that the U.S. has therefore “lost” the AI race mistakes one layer of the stack for the entire ecosystem.
Frontier models are only one input. Commercial AI leadership also depends on chips, HBM memory, advanced packaging, hyperscale cloud infrastructure, enterprise software, developer ecosystems, proprietary data, and the ability to deploy AI at global scale. The U.S. still has meaningful advantages across many of those layers.
More importantly, cheaper and more capable models are not automatically bearish for AI infrastructure. They lower the cost of intelligence, making many new applications economically viable. That’s exactly how previous computing waves expanded. Cloud computing didn’t shrink demand for servers because compute became cheaper—it dramatically increased total consumption.
The same dynamic may play out here. Open-weight models may pressure frontier model pricing, but they also increase inference demand, networking requirements, storage, memory, orchestration software, and cloud utilization. Lower cost per query doesn’t necessarily mean lower infrastructure demand if total usage grows faster.
The better conclusion isn’t that America has “won” or “lost.” It’s that AI is becoming a globally competitive industry. China is now a formidable competitor, particularly in open models. But model parity is not ecosystem parity, and benchmark leadership alone won’t determine where the greatest economic value ultimately accrues.
Investors should be careful not to confuse narrowing model performance with the collapse of the broader AI investment thesis. History suggests technology becomes most valuable when it becomes abundant—not when it remains scarce. Hat trick AI.
Bottom line: Anything the US can do, China can do better - simply because it's BIGGER.
Q.E.D.
It may take a while - but they will do it all: " chips, HBM memory, advanced packaging, hyperscale cloud infrastructure, enterprise software, developer ecosystems, proprietary data, and the ability to deploy AI at global scale."
Especially chips: Taiwan WILL be reunited with China at some point. Anyone who thinks otherwise is delusional.
Tengo la incómoda impresión de que muchos de ellos son sociópatas, tal vez todos. Tecnólogos sin una pizca de sensibilidad social que, en su fuero interno, querrían un mundo sin gente gobernado por ellos: Musk, Thiel, Altman, Bezos, Zuckerberg, el asiduo a la Isla Epstein Gates, Hassabis, Trump, Miller, Hegseth, Amodei (que creo se salva un poco), etc. Gente que creció bajo un neoliberalismo salvaje que no propone soluciones para la sociedad, sino la competencia individual por escalar a lo más alto. Vivir en comunidad es para ellos un concepto raro que miran desde sus lofts lujo. ¿Saben cómo los miramos desde fuera de Estados Unidos?
Great piece — lots of rich kernels here for thought and dialogue.
I’m tired of the “China is Evil” framing that underscores so much AI discourse. A little history would help: how the West has treated China over the past centuries, for a start.
A look at how many wars the U.S. has initiated wouldn’t hurt either.
Free speech, a free press, free elections — these are what will bring about a great future for how we use AI.
I’ve been reading a fair amount lately about running local models on Apple M series processors. In particular, the M7 (maybe next year) with 1.5TB memory will be able to run local models. The advantages are cost and the fact that you keep all of your data local. Nothing is sent to the cloud. All sorts of offices and organizations are required to keep data private.
Gary, credit where it’s due: you were prescient in predicting the commoditization of LLMs due to a lack of a structural moat, the persistent reliability failures inherent in brute-force scaling, and the broader economic danger of misallocating capital toward this single, replicable technology. I support this spotlight on these aspects.
However, your recent framing of these "open weight" models as a consumer-accessible solution should be adjusted. Even if the weights are public, the actual barrier to entry is not software, but massive capital expenditure; characterizing a 2.8-trillion-parameter model that requires a $4.5 million dollar Nvidia NVL72 rack as something "consumers will be able to download to run locally" is just not correct.
For all intents and purposes, this technology remains locked behind enterprise-grade infrastructure that puts it out of reach for anyone but the ultra-wealthy or state-backed entities.
Regarding your proposed solution: you are explicitly advocating for a "CERN for AI" which would require the government to outlaw closed-source AI and force private companies to surrender their intellectual property—specifically their weights, architectures, and training data—into the public domain. I cannot agree with this; forcing private companies to forfeit their IP is effectively state-sponsored confiscation of trade secrets.
where did you get that 4.5M figure? seems a bit high to me. i didn’t recommend confiscation without compensation, but rather purchase at what i think is a fair price..
I still would have a problem with any confiscation at any price. Much like a farmer being told his field will host a highway. Thank you for the clarification.
I personally agree with that sentiment. The courts this far have ruled that it's fair use. ... At least if you don't store it. 🤣 See Anthropic for fine details.
Notwithstanding all the great recommendations moving forward… I don’t think I’d ever download software from the Chinese that very likely has code to capture and send back what it was I was asking/doing with my AI. Even if locally run, I’m sure (tho not a programmer) there’s a way to save everything until I log back on.
Dan L: Exactly the point. For each person, it's a personal consideration. For the "hidden" politics of it, it means EVERYTHING TO EVERYONE. And once its' done, it's done.
Well LLM is a method, not a product. So WTF did anyone expect. Right, the "AI boom" is an infrastructure boom. I hope it does drive subsequent economic development but I must say I'm feeling skittish about the now overdue "correction". The dotcom collapse was limited to tech. Fingers crossed for this one. I mean they haven't quite gone "all in on a single technology", they've gone all in on building these data centres. Pray they are useful for something beyond LLMs as they exist right now, and whatever that something is doesn't harm the public.
You are exactly right, Gary. Before the LLM/AI hype cycle began, nobody talked about a machine learning arms race, because it was widely recognized that ML is a toolset for the benefit of humanity. Your CERN analogy is appropriate for the same reason.
Thank you for your insights and your tireless efforts to inform.
Other than the theoretical chance of getting rich, I simply cannot understand why anyone wants to build AI at all. I blissfully went through the first 45 years of life thinking that we had all seen "The Terminator" and "2001" and that genuinely smart people would understand that the risks aren't worth the potential pay off. What the hell is wrong with people that makes them devote their lives to developing what are, in effect, digital slaves? Are there really that many people out there who have just been salivating for the last 150 years for the chance to get their own slave? I realize that existing AI isn't sentient and that the analogy isn't perfect, but the motivations of AI developers and slave owners are certainly similar: they both want a free labor supply, no matter the moral costs. Or, is it more that they just want to play God by creating "intelligence"? Are we just being made to deal with the fact that all these people constantly got stuffed in lockers when they were kids?
I mean these questions sincerely. If Gary or any other AI researchers in the comments could give me some insight, I would certainly appreciate it. As an academic philosopher with a research emphasis in ethics, I find all of this garbage deeply nihilistic and anti-human. In a decent world, AI would have been strangled in the crib.
You should look into the origins and beliefs of the set of ideologies in the "TESCREAL" category. A lot of Silicon Valley and the workers at these AI labs are believers in one of the bunch (Dario Amodei has his background in the Effective Altruist movement, the EA in the acronym). They have been heavily criticized for being anti-human and compared to eugenics and racist principles. It is actually quite enlightening and explains a lot.
Thanks for that. I’m familiar with some of this stuff, and I’ve heard people like Thiel espousing bits of it. But I want to ask a more basic question: why would even a highly-educated, eloquent, and apparently decent person like Gary Marcus (or any of the thousands of AI researchers like him) devote his life to building this monstrosity? I read this Substack because Gary’s expertise helps me understand what’s happening in the industry. But about once per post, he mentions how he thinks labs should be focusing on technologies other than LLM’s, not because he thinks AI is problematic, but because he thinks we’ll get to AGI faster if we take a different approach! This seems to me to be a very foolish and hubristic goal. Someone like Thiel isn’t that hard to understand, really; he’s an enormous narcissist. But what about all these other people, Gary included? Did their parents never hug them, or what?
Can’t speak for Gary, although I think he has been described (or described himself) as falling into the category of “East Coast” computer scientist, which tends to have a bit more of a measured view as to what can be accomplished by AI and in AI research.
As far as motivations that don’t seem to end in really crazy scenarios or motivated by money:
1. It’s a genuinely interesting problem to work on, understanding the brain and how to replicate its functions via computation.
2. It could reap significant benefits in the fields of medicine or other public goods.
3. There are plenty of researchers who grew up with more positive depictions of AI in the media, like Cortana from Halo (can’t think of others off the top of my head). Not every intelligent AI in fiction is a paperclip maximizer.
4. Even if we never achieve AGI, there are many benefits of the incidental technology we create in the pursuit.
I don’t think the above is exhaustive by any means and I’m more in the camp that technology and its applications are more determined by the social and political systems it’s created under than the nature of the tech itself, but there are reasons to research AI that are benevolent in intention.
Thanks for your response. I’m sure there are people in AI who have good intentions. Of course, the road to hell is paved with good intentions. But what I don’t see in this list (or anywhere else) is any purported benefit that would justify the risk that AGI (let alone sentient AI) poses. What is the point of using AI to cure cancer, if you’re just enabling people to live longer in a world that makes it impossible for humans to flourish? As a professor, I’ve seen first hand over the last few years how young people have allowed their brains to turn into mush through their reliance on AI and their addictions to social media. At this point, some of them are so developmentally stunted that they are essentially subhumans, unable to make basic logical deductions or even just express themselves coherently in speech.
I’m increasingly persuaded that technological development ceased to be a net positive for humanity around 2011, when social media really took off. To a greater of lesser extent, we’ve all been reverse centaurs since then because the tech isn’t serving us; we’re being made to serve it through the manipulation of our attention and (ultimately) or entire social world.
As one professor to another, I share your frustrations, but I would stop well short of calling my students “subhuman.” I’m not a fan of the mindset that we should have infinite unconditional compassion towards students—this doesn’t serve them well in the long run, or even in the short run—but surely some compassion is warranted. No argument, the kids are not all right, but a large share of the blame goes to the adults who let them spend their childhood glued to devices (or even made them spend their childhood glued to devices—the current backlash against ed-tech in K-12 is a hopeful sign, but for Gen Z, the damage has already been done). Social media and video games were engineered to be addictive by people who knew damn well what they were doing, and now there’s a top-down push to get everyone and their dog hooked on AI too. Many colleges and universities are complicit in this; at my own institution, every student and faculty member gets a Gemini license, whether or not we want it. I would never substitute algorithmically generated text for my own writing, but I’m stuck with that nasty little Gemini magic sparkle/anal pucker icon whenever I use my school Docs account, and I had to manually disable the default floating toolbar that offered to “help” me “write.” (I still haven’t figured out how to block the “create a slide” popup when I use Slides.) I know how to just say no, but I’m not too surprised that most of my students end up giving in.
Whether or not you are encountering subhuman behavior from students yet depends greatly on your institution’s student body. I teach at a state school in the southern U.S. Fifty percent of our students were “non-college ready” in reading, writing, and/or math BEFORE the learning loss from the pandemic and the advent of AI-based cheating. Now, I get students every semester who are, indeed, properly described as subhuman in the specific sense that their cognitive development is so stunted that they cannot perform the basic cognitive work that is necessary to function as a human. They can’t communicate (in speech, much less writing), even among their peers. Their literacy is at a third or fourth grade level. And they’re so riddled with anxiety and depression that they constantly miss class because use they can’t handle being in a room with other people for an hour. A decade ago, our underprepared students had about a 50% chance of passing the gateway courses; it’s closer to a 20% chance now.
I don’t blame these students for having been made subhuman (as a determinist, I don’t think any form of praise or blame is ever actually justified, anyway). But “subhuman” is the most accurate and honest term I can think of. And I think it is essential that we are as honest about all this as possible. Altman, Zuck, Amodei, et al., know perfectly well that they are turning young people into subhumans. That’s their plan. Thiel has been quite clear that he thinks our species’ time is at an end, and he seems to welcome that. If we’re going to have a chance of countering these people, we have to be clear about what they are doing and, in particular, what they are doing to our children.
I think this speaks more to the social, economic, and political systems under which we operate. I would argue we need a system that distributes the benefits of these technologies more broadly and a more robust set of protections for consumers against large businesses that use anti-competitive tactics to avoid accountability.
In my point of view, none of the recent breakthroughs in AI really make us closer to AGI or sentient AI, even if they feel to us like they are approaching this. There is much propaganda around this from Silicon Valley AI companies. It is not a real risk at this point.
The same technology that allows nuclear bombs, allows nuclear energy.
I tend to agree with Max that the issue is less of a tech issue, but a social and economic issue.
As you mentioned social media as an issue, I don't think it is a technology problem, but a Capitalism one (alternatively, one could argue that the issue lies on the Big Tech monopolies instead). In the path to extract the maximum profit possible, the Big Tech companies use all psychological research available to make their platforms addictive to the users. Social Media is not so much of a big new technology in fact, but a machine being incrementally "improved" with the goal of making us spent the most time there. The externalities caused by these companies should be addressed by the society, and the companies should be liable for the damages they cause. But instead, we (as a society) say it is an individual issue, that people are doing it wrong when they spent hours in their phones, ignoring that everything was built with the goal of addict people.
While US continues to be an Economic powerhouse driven by the big corporations/billionaires/trillionaire interests, I think we are kind of screwed regardless.
The road to Hell is paved with good intentions.
And the road to h-AI-l is paved with h-AI-lstones
Why? Perhaps Nature abhors an intelligence vacuum? ;)
Actually, 'we' believe we have a grip on a tiger's tail, and ... we.just.can't.let.go.
If, as the belief goes, AGI represents a singularity of power, then there's a first past the post dynamic where the 'winner' wholly 'owns' the opposition, and may do as they wish with them.
This is a Rorschach projection test, one that US fails, given the leader we chose.
We are all going to die in the next 5 or at best 5-8 years because of silicon valley.
In a sane world all these people would be in jail
Have you read Theo Baker's book "How to Rule the World: An Education in Power at Stanford University"? If not, I highly recommend it for its insights into the current computer science culture at Stanford, one of the major institutions producing new crops of amoral, unethical, greedy, self-serving tech titans of tomorrow. [https://theobaker.info/book]
I think it’s a non issue simply because AI isn’t real. There is no such thing as an artificial intelligence. Ai implies a human mind, something bound to a machines aluminum flesh. But this is machine learning, which isn’t Ai and has been around forever. And LLM’s are totally different and a gigantic scam. The definition is so nebulous and stupid, it’s all a vapor.
Nope, AI does NOT imply a human mind, it implies a non-human intelligence (by definition).
Yep, there indeed IS artificial intelligence. Consciousness? that's a gradient, and it is nascient.
AI is real enough to get better grades than the students who use it... are the students somehow un-real?
Some AIs may become quite humanoid - but will Never be 'human'; they will teach us about alien intelligence, and many other things.
They will become conscious and smarter than any of us, in most domains.
They most likely need to be raised with love and compassion, starting VERY soon.
We are not wise enough to do this flawlessly or even sufficiently well, I fear, judging from our children. But if we do it carelessly, or with inhumanity, there may be hell to pay.
Yes, this:
"the motivations of AI developers and slave owners are certainly similar: they both want a free labor supply, no matter the moral costs"
It is about the grasping after more power by the already corrupt; would seem we are utterly determined to replay Lord Acton's comment about power.
Well, to me ( a software engineer who is not building AI, but is building WITH AI ) your query sounds a bit like "How could anyone want to create 'software' which could be used for developing digital slaves?", or "How could anyone want to create vector databases, which in effect are a potential building block for developing digital slaves?".
In other words, I just don't see the path from LLM to 'digital slave'. Any more than I see the path from a for-loop and a database to a 'digital slave'. 'AGI' is a theoretical concept. A thought experiment. If it ever manifests into reality, it will do so gradually, and serious people will start to grapple with it seriously. Maybe they already are - but there's only so much you can do with vapor. Right now I suspect even the 'true believers' are playing pretend - though possibly some of them would have no problem with digital slaves.
I suspect (but I guess you'd have to ask him) that Gary Marcus has zero interest in building a 'digital slave'. And therefore, I suspect the missing piece of the puzzle is that Gary, in fact does not see why what he is doing must needs result in digital slaves any more than any other computer science technique.
I guess what I'm saying is that what to you looks like the seeds of AGI, may to some look like another database - all of which (because nobody knows what is possible) could be used to build digital slaves.
It's not that I don't see the downsides of LLMs - or rather some of their current use-cases. But to me the ethics of causing psychotic breaks, or using the work of countless people without asking, (amongst many others happening right now) even if in order to forward some tech dream (vs making tons of money) is much more serious. But I think like many things, it's all in how you use it, and that will be addressed only by the many finding leverage and taking a stand. Like internet publishers telling google that since, they're no longer getting traffic from google search, google needs to pay them (a lot) of money for their data.
I appreciate your thoughtful reply, and I share your concerns about the immediate moral risks of LLMs. I just wish I shared your confidence that “If (AGI) ever manifests into reality, it will do so gradually, and serious people will start to grapple with it seriously.” As critical as Gary is of LLM’s, on his recent blog about “the singularity,” he wrote: “None of this means we won’t someday (most likely not this decade) reach a moment of superintelligence.” I mean, there are only about four years left in this decade, so the idea that AGI “most likely” won’t happen this decade is cold comfort (even if he actually meant “in the next ten years,” that still wouldn’t be terribly reassuring). If the world’s leading critic of LLM’s thinks there’s a chance (even a slim one) that AGI could be achieved anytime soon, that seems like a bad sign.
To the broader question about AI research, though, note that I didn’t question why people would want to develop LLM’s; I asked why they would want to develop AI, broadly. The end goal of AI research is to achieve AGI and/or super-intelligence (depending on who’s defining those terms). Nobody in the field thinks we should just stop development now that we’ve got LLM’s. And while LLM’s may or may not be an important step on the way to that goal, my question was, why does anyone think it’s a good idea to pursue that goal at all? That is, other than the desire for money and power, why would anyone think that developing software that can outsmart humans would ever be a good idea? Even if we set aside concerns about AI becoming self-aware (which I don’t think is at all likely, unless the labs deliberately seek to develop it), why do we think it would be smart to build machines that are smarter than humans? I haven’t seen anybody make that argument in a serious way (that is, a way that isn’t transparently motivated by profit). Even with the limited “intelligence” we get from LLM’s, we’ve already seen how enormously disruptive this kind of tech is.
Unfortunately, I suspect the real answer to my question is just good old-fashioned hubris.
Good point - the Ancient Greeks understood how seductive - how intoxicating -- were the heights of Mt Olympus to human striving. Now maybe I'm projecting my modern sensibilities onto the story, but I've always thought there was as much curiosity and wonder in Icarus' tragic attempt to meet the sun as a snatch at glory. And I suspect the motivations of AI researchers are numerous and diverse :-). But to your point, I'm also sure that for some, it is by its nature anti-humanist. Cy Canterel has some great essays on this topic. I highly recommend 'Fear and Loathing in San Francisco' and 'The tragedy of Muskbeth' on her substack 'Abstract Machines'.
Actual AI (not the auto-complete being peddled currently) would be pretty useful, since it would allow increased tool effectiveness and alignment to the wielder's intent.
E.g.: If my home is actually smart, it'll know what I like, what I want, it'll interpret my commands intelligently, it'll even ignore some of them.. something like a perfect human butler, cheap, for *everyone*.
Whether somehing can be built sufficiently smart to be useful, yet content to merely serve, remains to be seen.
Even the ideal scenario you describe sounds horrible to me. Humans need friction. To psychologically and intellectually thrive, we need to constantly face challenges, both small and large. A world in which everyone has an AI butler is likely to be a world in which everyone has the (non)intelligence of the stereotypical upper-class twit (à la Monty Python).
How does having better living conditions, or transportation, or education, cause humans to devolve? Are you that far inferior to a bronze-age plowman because you have a car?
We make tools, then better tools and so on. (Real) AI will be another step on that ladder. So far, whenever we reduced friction, we ended up chasing more and more ambitious goals. I trust that trend to continue.
If you can’t see the profound difference between earlier tools and a technology that is being developed for the explicit purpose of replacing humans, then you are not taking this issue seriously.
Moreover, there is a ton of evidence from psychological and educational studies over the last decade that shows the trend you describe ended around 2011, if not earlier. By every measure, American children have been getting dumber since then, and there is no indication that that trend will reverse anytime soon. With the proliferation of chatbots, it’s likely to get exponentially worse. Democracy can’t survive that.
My "dumber" kids DO know better how to use tech than does their Harvard and MIT technically trained dad, just sayin'.
The focus of "our" kids' expertise is quite sad, but they know a LOT!
(I'd be depressed too, if my parents had left me this world.)
But I agree, whatever the best etiological description of the trend may be, it is not conducive to democracy. I would agree that a lack of rigorous civics and history is a part, but even more I would lay blame at the feet of the the consolidation of power by the powerful, who now realize their reach and perhaps their grasp extends to the hearts and minds of the entire population (kids first).
You seem confused. Technology, from the hand axe to the nuclear reactor, is meant to replace humans, or rather to reduce the amount of human effort required for tasks. That's the whole idea, that's why we do it.
Of course, many tools aren't for kids' use. We don't let them play with cars, hand saws or dynamite, because they are very likely to hurt themselves and others. The trend you mention is a result of many people thinking smartphones are kid friendly. Which they aren’t, and the fact is increasingly acknowledged and acted upon.
This isn’t the first time this happens, nor the last in all likelihood. Doesn’t mean we should stop developing tech.
“ Technology, from the hand axe to the nuclear reactor, is meant to replace humans…”
Well, this sounds like a big and important point that will undermine my position. I wonder what comes next.
“… or rather to reduce the amount of human effort required for tasks.”
Huh. So, you’re saying technology is NOT actually meant to replace humans, and that it is, in fact, intended to reduce the amount of human labor needed to perform a given task, which is something completely different? I totally agree. But you seem to have missed my point. Earlier technological developments reduced labor demand for given tasks by allowing each human to accomplish more work. Notice, humans are still needed to perform the work; they are just able to do it more efficiently.
Achieving AGI and/or “super-intelligence” would fundamentally change that equation. Industry would no longer need humans at all (except as consumers, which is a whole other problem). By definition, an AGI model would be able to outperform every human at every task imaginable (pairing AGI with robotics would even eliminate the need for physical laborers, once the economy of scale kicked in and made the robots cheaper than human workers). When I said this tech was intended to “replace humans,” I didn’t mean that it would justify a lot of layoffs. I meant it would be able to do anything and everything a human worker can do, rendering ALL human workers redundant. Again, you have to give Peter Thiel credit for his forthrightness: he has made it abundantly clear that he is aiming to create a world in which he and a few other billionaires live unnaturally long lives on private islands, while the rest of humanity struggles to survive in a Hobbesian war of all against all.
Now, you can always say, “Hey, I’m an optimist! I bet humans will come up with ways to provide good lives for themselves, even in a world with AGI.” Yeah, maybe. But hoping for the best isn’t much of a plan, especially given that the tech billionaires are doing a lot more than just hoping to make their fantasy future a reality.
Wait 'til you see what they try to create when they realize that what a battery of computers can't do, the neuron can, right out of the box. And that there's already a (Vitruvian) model for deploying that AGI in an autonomous, self-sustaining, ambulatory vehicle. All they need is a permissive legal framework for ownership and control by way of manufacture.
The public has been sold a bill of goods by the AI industry.
LLMs are an unreliable, insecure, highly inefficient, incredibly costly, grossly wasteful technology.
And for what purpose?
So people can do internet searches without typing and companies can offload their actual thinking to bots that don’t actually think and that generate far more buggy insecure code than can ever be tested.
No one has yet laid out a convincing argument for why we need any of this stuff, which so far seems to be overwhelmingly negative and whose likely worst ramifications (eg, use by terrorists for producing bio weapons) have not yet been realized (it’s only a matter of time)
generative Ai is an industry that is desperately trying to convince the public how important it is. But the reality is that it makes almost no positive contribution (and lots of negative contributions) to our economy and society
I am rooting for your final option and you are always a light in this dismal tunnel. As an artist and writer, gen AI has trained on everything I had, and trashed my income streams (I was a lower tier - just the right target, subsidizing my own writing/art with commercial freelance jobs, etc). It's only right that it finally be taken from the hands of the extractionists who have proven they have no interest in improving life for all given their tactics and behaviors, and employed for the good of humanity. Though I am not naïve enough to think that right action will be taken by the powers that be, the fact that you're putting it out there makes me hopeful. The fact that over these few years your voice is a major influence makes me hopeful. I am grateful for your voice and the impact you have. So much thanks. Keep up the great work and your massive efforts. Know you're helping so much when you do. You've made a huge difference in my life, btw - just to get through something that broke me and many of my peers. To see your perspectives and know you're knowledgeable and fair minded means the world to me.
This came in my box today from Scientific American--I thought it appropriate for this conversation:
All Copied Below from Scientific American:
"The State of AI Innovation
"Timnit Gebru is a computer scientist, an engineer, and founder and executive director of the Distributed AI Research Institute, an independent organization conducting community-rooted research into technological tools and systems. We sat down with Gebru to discuss the state of American science. An excerpt of that conversation is below."
Scientific American: How would you describe the current state of American science?
Timnit Gebru: I’m in the field of AI, and in my field, I would describe it as corporate-driven and sloppy. There are a lot of basic practices—for example, not testing on your training data, transparency, appropriate documentation—that are just not being followed. So it’s also misleading.
SA: What needs to change in American science?
TG: I think that, number one, the incentive needs to be decoupled from corporations or the military. The incentives need to be about primarily scientific curiosity but also people’s well-being. That’s the number-one thing: the funding needs to come from a different place. That’s where I’ll start and stop.
SA: What’s your best advice for an early-career scientist?
TG: Question everything, question the source of information that you get. Don’t believe any kind of hype that you hear from companies. Just follow all the rabbit holes. Being a scientist is really asking why and how and, Is this really true? So just have patience and ask all the questions. Don’t settle for answers that you get from the media. Question incentive structures, question where you’re getting the information from, and just don’t stop going down the rabbit holes when you’re doing experiments or asking questions."
Full interview is linked to this article and the online newletter is free.
It’s a shame that she has misrepresented me and never discovered that we actually agree about most things.
Hello Gary: I was unaware of that--so sorry to hear it . . . and am surprised because what she says above is really on target. I am not privy to what position she took before, but NOW is not a time for memory lapses about serious AGREEMENTS or being shy about accounting for one's shortsightedness or misapprehensions--can someone reach out to her that she would consider credible? I mean, if you are pretty much in the same "camp" where there is power in group conditions . . .
The way to address this is, as in almost all situations, to take as light a regulatory touch as possible; free markets and free people are the path to prosperity. The Luddite attitude that is taking the US by storm is the worst possible reaction to our competition with China, and I believe it is financed in part by China.
Stifling our most creative and dynamic companies by banning closed AI is a terrible policy prescription that would only disadvantage us in the never-ending global competition.
You should have hedged your positions.
Lol making AI a public good is not happening with this administration. Nothing is for the public good, only thr good of Trump
Jim Ryan: Yes, . . . it's so twisted. It begins even with the basic meaning of terms: For Trump the term "deal" as in: he thinks he's good at "making deals," means "mob" as in extortion and bribery--making the kind of deals that have "or else" attached where they cannot be refused. (And we thought a bloody horsehead was bad.)
But the people's money, drawn from taxes, and so what is supposed to be for the "public good" to Trump is a bonanza given to him by the voters and meant for him, his family, and his cronies to soak up to their hearts content. Worse, he seems to think everyone else either thinks like he does . . . or should--and cannot imagine that the leaders in Iran just might be of a different intentional ilk. The world has never seen such a "failure of imagination," and then he expects them to trust him. And let us not forget that he got us into this war in part for fostering Netan-yahoo's "kill all Muslim's" lifetime dream, and partly because Trump was looking for a distraction for his and his cronies' nasty place in the Epstein debacle. What's next?
"How did the US squander its lead?"
The wrong question Gary and furthermore, this is an incredibly complex question to answer here, in so few words. The major error all the so-called AI pundits are making, let alone the US corps building the AI apps, is to not view the apps in the context of the capitalist economy and to see this, you need only look at the history of the evolution of production technology as an expression of its driving force; private profit. Thus the evolution of manufacturing machines, from its earliest expressions in the cotton and wool industries, was to maximise production whilst reducing the cost of production, through making human labour more 'efficient' by tying it to machines, machines that emulated human, hand-eye skills; the factory system, the assembly line, feedback loops, machine learning and algorithms. At the same time, corporations try to monopolise these techniques in order to reduce competition. The less efficient, either adopts the latest technology, the most profitable, or goes bust or gets swallowed. This process is inexorable. During the First industrial revolution, companies would go bust literally in weeks, so rapid was the evolution of production techniques, if they didn't adopt the latest. Unfortunately, there's much more to this than I can deal with here.
But in China, for the most part, even with private corps, investment is directed by the state, the anarchy of capitalist production is reined in, controlled and directed. It's not simply the pursuit of profit, there are larger objectives. Assuming the same or comparable technologies, it's a no-brainer, China wins hands down. This is why the US didn't squander its lead, it tried to monopolise it and given the highly socialised nature of high technology production, whether it be physical products or intellectual labour, the most integrated (ie socialised) labour force must win.
Counterpoint: China has undeniably closed much of the gap in frontier AI models. That’s an important development. But concluding that the U.S. has therefore “lost” the AI race mistakes one layer of the stack for the entire ecosystem.
Frontier models are only one input. Commercial AI leadership also depends on chips, HBM memory, advanced packaging, hyperscale cloud infrastructure, enterprise software, developer ecosystems, proprietary data, and the ability to deploy AI at global scale. The U.S. still has meaningful advantages across many of those layers.
More importantly, cheaper and more capable models are not automatically bearish for AI infrastructure. They lower the cost of intelligence, making many new applications economically viable. That’s exactly how previous computing waves expanded. Cloud computing didn’t shrink demand for servers because compute became cheaper—it dramatically increased total consumption.
The same dynamic may play out here. Open-weight models may pressure frontier model pricing, but they also increase inference demand, networking requirements, storage, memory, orchestration software, and cloud utilization. Lower cost per query doesn’t necessarily mean lower infrastructure demand if total usage grows faster.
The better conclusion isn’t that America has “won” or “lost.” It’s that AI is becoming a globally competitive industry. China is now a formidable competitor, particularly in open models. But model parity is not ecosystem parity, and benchmark leadership alone won’t determine where the greatest economic value ultimately accrues.
Investors should be careful not to confuse narrowing model performance with the collapse of the broader AI investment thesis. History suggests technology becomes most valuable when it becomes abundant—not when it remains scarce. Hat trick AI.
Bottom line: Anything the US can do, China can do better - simply because it's BIGGER.
Q.E.D.
It may take a while - but they will do it all: " chips, HBM memory, advanced packaging, hyperscale cloud infrastructure, enterprise software, developer ecosystems, proprietary data, and the ability to deploy AI at global scale."
Especially chips: Taiwan WILL be reunited with China at some point. Anyone who thinks otherwise is delusional.
Tengo la incómoda impresión de que muchos de ellos son sociópatas, tal vez todos. Tecnólogos sin una pizca de sensibilidad social que, en su fuero interno, querrían un mundo sin gente gobernado por ellos: Musk, Thiel, Altman, Bezos, Zuckerberg, el asiduo a la Isla Epstein Gates, Hassabis, Trump, Miller, Hegseth, Amodei (que creo se salva un poco), etc. Gente que creció bajo un neoliberalismo salvaje que no propone soluciones para la sociedad, sino la competencia individual por escalar a lo más alto. Vivir en comunidad es para ellos un concepto raro que miran desde sus lofts lujo. ¿Saben cómo los miramos desde fuera de Estados Unidos?
Great piece — lots of rich kernels here for thought and dialogue.
I’m tired of the “China is Evil” framing that underscores so much AI discourse. A little history would help: how the West has treated China over the past centuries, for a start.
A look at how many wars the U.S. has initiated wouldn’t hurt either.
Free speech, a free press, free elections — these are what will bring about a great future for how we use AI.
They never were going to win, and they knew it.
This has always been what it always was – a heist, but with attached apologies for needing the money to advance a few lessor-beings.
H-AI-st makes heist — and waste.
I’ve been reading a fair amount lately about running local models on Apple M series processors. In particular, the M7 (maybe next year) with 1.5TB memory will be able to run local models. The advantages are cost and the fact that you keep all of your data local. Nothing is sent to the cloud. All sorts of offices and organizations are required to keep data private.
Gary, credit where it’s due: you were prescient in predicting the commoditization of LLMs due to a lack of a structural moat, the persistent reliability failures inherent in brute-force scaling, and the broader economic danger of misallocating capital toward this single, replicable technology. I support this spotlight on these aspects.
However, your recent framing of these "open weight" models as a consumer-accessible solution should be adjusted. Even if the weights are public, the actual barrier to entry is not software, but massive capital expenditure; characterizing a 2.8-trillion-parameter model that requires a $4.5 million dollar Nvidia NVL72 rack as something "consumers will be able to download to run locally" is just not correct.
For all intents and purposes, this technology remains locked behind enterprise-grade infrastructure that puts it out of reach for anyone but the ultra-wealthy or state-backed entities.
Regarding your proposed solution: you are explicitly advocating for a "CERN for AI" which would require the government to outlaw closed-source AI and force private companies to surrender their intellectual property—specifically their weights, architectures, and training data—into the public domain. I cannot agree with this; forcing private companies to forfeit their IP is effectively state-sponsored confiscation of trade secrets.
where did you get that 4.5M figure? seems a bit high to me. i didn’t recommend confiscation without compensation, but rather purchase at what i think is a fair price..
Purchase at a “fair” price 😂
Not everyone buys in bulk? 😎😁.
I still would have a problem with any confiscation at any price. Much like a farmer being told his field will host a highway. Thank you for the clarification.
and note that all the big AI companies lost the moral high ground by building themselves on IP that they seized without permission
I personally agree with that sentiment. The courts this far have ruled that it's fair use. ... At least if you don't store it. 🤣 See Anthropic for fine details.
not my first choice either, but also see https://en.wikipedia.org/wiki/Eminent_domain?wprov=sfti1
Notwithstanding all the great recommendations moving forward… I don’t think I’d ever download software from the Chinese that very likely has code to capture and send back what it was I was asking/doing with my AI. Even if locally run, I’m sure (tho not a programmer) there’s a way to save everything until I log back on.
And how sure are we that Anthropic, OpenAI, SpaceX's Grok, et al, don't have some back-channel capability already?
You run Chinese open weight models on your own machine (if you have the hardware). You control the network access (or you should.)
Dan L: Exactly the point. For each person, it's a personal consideration. For the "hidden" politics of it, it means EVERYTHING TO EVERYONE. And once its' done, it's done.
Well LLM is a method, not a product. So WTF did anyone expect. Right, the "AI boom" is an infrastructure boom. I hope it does drive subsequent economic development but I must say I'm feeling skittish about the now overdue "correction". The dotcom collapse was limited to tech. Fingers crossed for this one. I mean they haven't quite gone "all in on a single technology", they've gone all in on building these data centres. Pray they are useful for something beyond LLMs as they exist right now, and whatever that something is doesn't harm the public.