The New Sport of Misrepresenting AI Criticism

AI has undeniably made progress; systems like DALL-E and Stable Diffusion are impressive, surprising, and fun to play with. We can all agree on that.

But can the current state of AI be criticized? A lot of recent media accounts have tried to stick AI critics in a little box. Scientists like the linguist Emily Bender and the computer scientist Melanie Mitchell have rightly pushed back on that.  Mitchell, for example, wrote yesterday, and I agree with every word:

Melanie Mitchell @MelMitchell1

Like in all sciences, it's really important to have constructive debates about AI research, and to have scholars who write critically about specific results in the field . It's not helpful to brand these scholars quite generally (and negatively) as "AI Critics" or "AI Deniers".

What provoked Mitchell’s tweet? In no small part it was (as she acknowledged later) a recent Substack post by the writer and scientist Erik Hoel, “How AI's critics end up serving the AIs”, which claims in its subtitle to be discussing “the obscuring effects of AI criticism”. Needless to say, a lot of people don’t like AI criticism, and the essay has gotten a fair bit of traffic. If you are impressed with current AI and don’t want anyone to spoil your fun, I recommend it.

§

Most of Hoel’s criticism is directed at me. (“I single out Marcus as representative of AI critics everywhere”.)

From the outset, I could tell that the 4,200 word long piece was not going to be fair. In the introduction, for example, Hoel tries to paint me as a dilettante, “Marcus hit on this topic [AI] after searching around for a few different ones over the years, ranging from a nonfiction book on how he learned to play the guitar to his book on evolutionary psychology, Kluge”, entirely neglecting the fact that my 1993 MIT PhD dissertation was on child language acquisition, generalization, and neural networks – which is exactly what I am writing on today, nearly 30 years later. He also neglects my book The Algebraic Mind, a 2001 book that Luis Lamb and Artur Garcez credit as a major inspiration for their own pioneering work on neurosymbolic AI.¹ (Yes, it’s true I spent my 2009 sabbatical learning to play guitar, and wrote about it. Sue me.)

After Hoel mangles my biography, he proceeds to mangle my views.

Part of his strategy (common in criticism of those who dare suggest that AI has not been solved) is to accuse me of goal post shifting - without even once actually pointing to a goal post I have shifted. In fact, to the contrary, the only sign he seems to give of what my goal posts allegedly used to be is this excerpt from a passage that I wrote in 2012 for The New Yorker:

[deep learning systems] lack ways of representing causal relationships (such as between diseases and their symptoms), and are likely to face challenges in acquiring abstract ideas like “sibling” or “identical to.” They have no obvious ways of performing logical inferences, and they are also still a long way from integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used.

For goodness’ sakes! Does he really think I abandoned those goal posts? Let’s look at the last Substack essay I wrote, um, all of 4 days ago:

Form, function, and the giant gulf between drawing a picture and understanding the world

The argument I made there is that … systems like DALL-E don’t “understand what wheels are—or what they are for”, because (as I discovered on Sunday) they systematically fail to properly illustrate requests like these:

If that’s not an example of exactly what I pointed to a decade ago — an illustration of deep learning systems lacking way of “integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used”, I don’t what is. Hoel’s goal-post shifting accusations are absurd.

I stand by the bit about logic too, and highly highly recommend a recent article by Guy van den Broeck’s lab at UCLA that shows that apparently successful performance on logic by neural networks is often driven by unstable statistical contingency, apparently reaffirming the importance of another of my 2012 goal posts. As van den Broeck’s team put it, their results “unveil the fundamental difference between learning to reason and learning to achieve high performance on NLP [natural language processing] benchmarks using statistical features.”

The problem for the neural network community is not that I have changed my goal posts, it’s that my 2012 concerns still stand. (Quite of the few examples that Ernie Davis and I mentioned in Rebooting AI go back a long further, to the 1970s.)

It may be boring to hear about the same challenges over and over, but it’s hard to see how we will get to trustworthy general intelligence until these problems are resolved, in ways that are robust, rather than merely statistically contingent.

§

In the only really substantive part of Hoel’s critique, Hoel tries to make it sound like I think that human beings are the measure of all things, which is something I have never said. In the evaluation of computational models, I have always been careful to distinguish two questions (and would urge the fields of AI and cognitive science to do the same):

• Is a particular model sufficient for its engineering purpose (e.g., for reasoning about the physical world)?

• Is a particular model a good model of human beings?

Over and over I have stressed that you could have a good model of engineering without being a good model of humans, or vice versa. A good model of the average human’s mathematical abilities would be a lousy calculator, and a good calculator would be a lousy model of the average human’s mathematical prowess.

Hoel tries to pretend that I don’t know the difference. Whereas I often say stuff like this, emphasizing that good engineering does not have to mimic humans:

Gary Marcus @GaryMarcus

@elonmusk @WholeMarsBlog This logic still makes no sense @elonmusk 👉Birds flap their wings, but that does not mean that wings are the only way to achieve flight. 👉Current neural nets lack the cognitive depth and robustness that would be necessary to support reliable L5 self-driving form vision alone.

I did recently ask whether GPT-3 is a good model of a human, but that’s different question, a scientific question about cognitive modeling, rather than a question about how to build AI.

§

The place where Hoel founders mostly deeply, though, is not on my personal biography or intellectual history, but on the question of the relationship between benchmarks and underlying cognitive mechanisms—a bit surprising for someone who is apparently trained as a neuroscientist (though not uncommon in people without scientific training). The heart of Hoel’ss argument boils down to (my words not his) “AI has done better over time on benchmarks so we must be making progress”; in reality, progress can be and often is illusory.

The naive view is that if a system does well on a benchmark that is designed to solve X, the system has actually solved X.

Over and over, the field of AI has discovered otherwise. Guy van den Broeck’s study is a perfect example of this. The Turing Test is another (succeeding or improving on it doesn’t necessarily mean you are intelligent; ELIZA did pretty well, and knew nothing).

§

Most active researchers have by now learned to tell the difference between what a benchmark says on the tin and what it is really getting at; in my Next Decade in AI essay, for example I quoted a team of Facebook AI researchers as saying

"A growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets... instead of learning meaning in the flexible and generalizable way that humans do."

Hoel is sure that AI has made progress because it has done well on a bunch of benchmarks. But the smart money is still on being cautious about this, and in recognizing that what the Facebook researchers said in 2019 is still the case.

That’s why people like MetaAI’s chief AI scientist Yann LeCun (who deeply disagrees with me about some foundational issues like innateness still thinks AI needs fundamental paradigm shifts and agrees with me that language models aren’t learning enough about the world. Even Geoff Hinton, arguably my staunchest critic, would agree that benchmarks aren’t cutting it, and that too much emphasis on them has stifled innovation:

One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas

Hoel could be smarter than me and LeCun and Hinton and all the rest of us, but a little modesty might not have hurt. Framing concerns as mostly unique to me when they are shared by the likes of LeCun and Hinton is rhetorical nonsense.

Those who wish that AI criticism would go away are welcome to their views, but misrepresenting critics and their views (and obscuring the degree to which even mainstream researchers agree) won’t magically solve the problems AI has yet to solve.

1. Hoel also seems to know that I went to Hampshire College, but not (though it is a matter of public record) that I was given early admission there (after 10th grade) because I wrote a Latin-English translator, one of my first AI efforts. I suspect that I have been thinking about AI for longer than Hoel appears to have been alive, quite to the contrary of his false and irrelevant accusation of dilettantism.