Hard-forked! Casey Newton’s distorted portrait of Gary Marcus and AI skepticism

What Casey Newton’s new essay skewering AI skepticism does—and does not—get right

“That was wonderful. I love being reduced to a cultural stereotype.”

— Carol Kane, in Annie Hall.

Casey Newton, perhaps best-known for his New York Times podcast Hard Fork, and author of the sometimes excellent Platformer, works fast. I got an email from him yesterday afternoon, sent him a bunch of notes, and two hours later he posted a critical essay on AI skepticism, largely targeted towards yours truly, with lots to think about.

Unfortunately, it was also perhaps among the most poorly considered essays he has ever written. (As discussed below, I am far from alone in feeling this way.)

To be fair, some of it is dead-on; but many other parts are seriously off the mark, about me, about the field of AI skepticism as a whole, and about where the field is with respect to AI.

Since there is a lot at stake, and Newton is widely read, it’s worth going into detail.

What Newton gets right:

• We ought to prepare – now – for AI to get a lot more dangerous than it already is. Many of us now suspect LLM returns are diminishing, but we should hedge our bets. Personally I’d bet a lot on returns on LLMs diminishing now or very soon, but I wouldn’t bet the entire species on it. If AGI dropped tomorrow, given the almost total lack of regulation with real teeth, I seriously doubt we as a society would be ready for it. If there is even a 5% chance I am wrong about diminishing returns, we still ought to be moving immediately to prepare. Even if the AI acceleration we saw from 2019-2023 slows down for five years, as I have often speculated, it won’t slow down forever. We need, for example, a global treaty, and those tend to take a decade. Let’s get moving, people.

• Part of the danger comes from the fact that we are very far from any guarantees that any sort of advanced AI (or current AI) that we can figure out how to build will be helpful, harmless, and honest. Newton and I are in agreement on that. As I said in a blog that he quotes approvingly “we still really cannot guarantee that any given system will be honest, harmless, or helpful, rather than sycophantic, dishonest, toxic or biased.”

• AI skepticism itself is worth considering as a kind of intellectual movement. People are going to have different views about its role in society, and it’s fair game for people to ponder whether we need more of it or less of it, how it could be improved, etc. As AI comes to have more power in society, AI skepticism becomes an increasingly important counterweight, and we should all want AI skepticism to be done right. It’s ok to be skeptical about skeptics, as part of a Hegelian dialectic, and I (as one of the better-known AI skeptics) don’t mind being dissected in that light (though as noted below, Newton’s focus almost exclusively on me is misleading).

• AI needs substantive regulation. (Here Newton writes, “Marcus also sat for an interview with the Wall Street Journal this week in which he made his case against the current generative AI models. I agree with much of it, including that AI needs a dedicated regulator.”) Neither of us are any too fond of having a fox like David Sacks guard the henhouse.

Where Newton is off

• Newton says (without sourcing) that it is “fun” to be an AI skeptic. Um, not so much. I will admit that GPT gaffes sometimes make me chuckle, and it was cool that time that John Oliver used one of my tweets, but on the whole this has not been a fun ride. I’ve been subjected to extensive ridicule from many of the major players in the field (Altman, Brockman, Musk, LeCun, etc), and that engenders a lot of hate mail from their many followers. I wouldn’t wish that on anyone. And the derisive tack that AI boosters have taken to insulate themselves from criticism matters. As Emily Bender has noted, the wild over-enthusiasm for LLMs have sucked the oxygen from the rest of the room.

• Newton vacillates far too much between AI and Generative AI, when in fact the whole ballgame is whether the $200+B investment in Generative AI is the right path to AI or not. For example, he gives a long list of things AI has done, but says very little about how those systems work; some are based on LLMs and Chatbots, but others are (I suspect) special purpose models that either don’t use generative AI or use generative AI as just one component in a larger system. The whole scaling debate (and the discussion around economics that Ed Zitron and I each often raise) is really about whether generative AI in particular can be made into AGI just using more data and compute; a system like AlphaFold 3 that combines classical AI with generative AI is on a different axis that has seen far less investment. It confuses matters to conflate all that together. Newton just doesn’t seem to get this. His writing on social media strikes me as astute; here he seems to be missing a central point.

• He gives a long, flawed list of of AI positives but scarcely takes note of the negatives, like the way it has been used for misinformation, for scams, for denying insurance claims, and so on; recent studies on covert racism in generative AI and the ability of generative AI to implant false memories are never mentioned. (Similarly my own credentials are given short shrift; there is no mention of my six books, such as Rebooting AI, which Forbes said was a must-read, or Taming Silicon Valley, which the New Yorker put on its 2024 best books list; my Senate testimony is never mentioned, etc, and he never engages deeply in arguments I have made around LLM limitations, nor does he consider in much depth the fact that CEOs with deep inside access like Satya Nadella and Sundar Pichai have recently reached similar conclusions. Newton studiously avoids giving me credit for seeing some important things early.)

• In his title, abandoning all pretense at journalistic objectivity, Newton calls AI skepticism, “phony comforts.” I don’t think that such derogatory phrasing can be justified. I write what I believe, often at great personal cost, and give detailed arguments for what I believe, sticking to it despite hostility. Much the same holds for many AI other skeptics. I have been saying what I believe about neural networks publicly since 1992, long before anyone in the public cared about AI. The recent statements from Andreesen, Nadella, Pichai, etc. bear out what I foresaw. (Perhapsthere is a second reading here of what Newton meant by “phony comforts” in that some unnamed set of people dismiss AI risks based on criticisms of LLMs such as mine; this too seems silly, given that few of us AI skeptics are known for comforting people. To the contrary, AI skeptics tend to document risks, and criticize others such as LeCun who downplay those risks.)

• Newton doesn’t seem to get the difference between an LLM improving on a specific example and failing on a general class of errors. Ernest Davis and I wrote in 2020 in Technology Review that GPT-3 was “a fluent spouter of bullshit” that had trouble with causality, physical reasoning, psychological reasoning, mathematical reasoning, and so on. All that’s still true, even if LLMs can now get the specific examples we published (and which are now presumably in the training set). Not one of those areas of weakness has been remotely robustly solved.

• The whole essay hinges on a false dichotomy, “It’s fun to say that artificial intelligence is fake and sucks — but evidence is mounting that it’s real and dangerous”. Here Newton has fallen into a familiar trap: thinking that if an AI is stupid (or overrated) it can’t be dangerous. As I have written multiple times before this is not so. For example, a stupid AI that is blind to facts can be misused to create propaganda, or create deepfakes. Likewise, an AI with a known tendency to hallucinate things can be used to target weapon (this as may happen soon, given the recent announcement that OpenAI will collaborate with Anduril).

• Newton deeply and doubly misrepresents me in this key passage:

““Therefore, superintelligence is unlikely to arrive any time soon, if ever. LLMs are a Silicon Valley folly like so many others, and will soon go the way of NFTs and DAOs.” This is a view that I have come to associate with Gary Marcus.”

This is NOT actually my view, and Newton should know better. I always say that AGI will come, just not soon, and in fact said so explicitly in emails to Newton yesterday (e.g., “we will get to AGI, but the field has been pursuing a hypothesis that was basically (just as I anticipated) disproven, so we will need new ideas.”) I also always say we need hybrid systems that include LLMs as one tool among many, never that that LLMs will altogether disappear, even if they become a commodity and valuations fall. The view he has “come to associate” with me is not remotely my view.

•  Newton also misleadingly tries paint me into the “AI is fake and it sucks” camp for most of the essay (though if you read very carefully, you can see that he walks that back briefly, because he knows full well that the view “he has come to associate with me” isn’t actually what I believe.¹).

For clarity, what I actually think is:

(1) Of course AI is real. I use speech recognition, GPS navigation, web search, and recommendation engines almost every day. A whole lot of students (perhaps the majority of OpenAI’s users) use LLMs daily to write term papers; coders use it a lot too, as I often note.

(2) LLM’s in particular are wildly overrated. I personally almost never feel the need to use them, though I understand some people have uses for them. They do not deliver remotely what was initially promised of them. But I can’t recall saying they “suck”; Yann LeCun said that. (Later Altman himself said that GPT-4 did.)

(3) LLMs are already dangerous, despite their limits. Covert racism? Deep fakes? Propaganda? Discrimination in employment, insurance and housing? Many downsides are already heere, and potential misuse in the military seems imminent.

• Newton tries to pin the AI skepticism tail almost entirely on me, downplaying my credentials and portraying me falsely as a lone voice, but there are many others who have challenged LLMs like Abeba Birhane, Emily Bender, Chomba Bupe, Ernest Davis, Ed Zitron, Melanie Mitchell, Margaret Mitchell, Carissa Veliz, Sasha Luccioni, Kate Crawford, Brian Merchant, Joanna Bryson, Safiya Noble, Missy Cummings and Meredith Broussard to name a few. Not one of them is mentioned. It’s a time-honored rhetorical trick to equate a field (here, AI skepticism) with an individual, trying to hoist that individual by some petard or another. (Scott Alexander, who Newton cites approvingly, pulled exactly the same trick; my reply then still applies today just as much.)

Don’t take my word for it

Since I am the target you might worry that I might be biased. And maybe I am. But, importantly, I am very far from alone in intensely disliking this particular Newton essay.

On Bluesky last night the responses were absolutely scathing, many harsher than my own. To take just a few examples, the journalist Ed Zitron called the essay’s rhetoric “disgraceful.” Fabio Chiusi of AlgorithmWatch wrote “I'm surprised how bad this piece is.” An anonymous account observed that Newton’s essay was “ wildly dismissive”. Professor Catherine Flick was no less complimentary, writing “I'm actually really cranky about the way Casey portrayed @garymarcus.bsky.social and the field that rightfully questions the hype, legitimacy and real world impact of LLM systems. It was a classic straw man fallacy.”

Four other people’s takes on Newton’s essay on AI skepticism

One patient reader named Timothy Faust went point by point through Casey’s Newton’s list of putative positives:

In fairness, though, not every Newton critic made it to the end of his essay:

Bottom line

Casey Newton’s essay made for an entertaining narrative, but the story he told is neither accurate nor balanced. It was more tall tale than trenchant criticism.

In a pithy DM summing it all up, a colleague said that that Newton’s basic strategy was to reduce me to a “caricature he's created … and then use that caricature to dismiss the whole field.”

§

Newton is of course right that we need to prepare for AGI, whenever it comes. But being a skeptic, especially one that aims to write with nuance, is no picnic; the attacks and misrepresentations never really end. Newton has added to them.

It is entirely possible to (a) be skeptical that we can simply scale LLMs to AGI, (b) be doubtful of the overall business model for LLMs given the lack of moat etc, and also (c) think that these tools are already powerful (though unreliable and untrustworthy) and therefore dangerous.

Newton’s essay would have been a lot more useful if it had been directed to that steelman, rather than the strawman he unconvincingly tried to pin on me.

Gary Marcus wishes that discussions around AI focused on actual, nuanced views.

1. In a disgraceful act of misleading rhetoric, Newton briefly acknowledges that I don’t really believe the views that he “most associates” with me, but still holds me responsible for a view that I don’t hold because the views that I hold might be useful to a set of unnamed people, who hypothetically might take solace in a misrepresentation of my views. (“Marcus doesn’t say that AI is fake and sucks, exactly. But his arguments are extremely useful to those who believe that AI is fake and sucks, because they give it academic credentials and a sheen of empirical rigor.“) In conjunction with the “most associate” with Marcus bit, this butt-covering paragraph is roughly on a par with saying “Taylor Swift doesn’t actually endorse murder, but I bet some murderers dig her music”.