Gaslighting and reality in AI

Some things that have and haven’t changed over the last few years

In fall of 2019, GPT-2 had started to grow popular and I started writing about it, a whole thread of examples of how it can easily be provoked into saying dopey things, and how superficial its understanding of reality was.

Gary Marcus @GaryMarcus

robust language understanding is about building discourse models across multiple sentences (see eg rebooting.ai & recent thread by ⁦@mpshanahan) on simple word problems like these, GPT-2 did not succeed once in 30 min testing my 6-yr old son scored 100%

Yann LeCun tried, at that time, to minimize my objections, saying they were just about math, in machines that weren’t trained to do math, with nothing deeper was amiss:

Yann LeCun @ylecun

@StanDehaene @GaryMarcus Actually, machines that are trained to deal with quantities do learn to deal with quantities. Gary merely says that machines trained to predict missing words don't learn much about quantities. Duh!

I said the problem wasn’t math per se, but that large language models didn’t represent robust world models of how events fold over time; I stand by this 100% even today.

Gary Marcus @GaryMarcus

Key problem with systems like GPT-2 is not that they dont deal with quantities (as @ylecun suggests below), it is they don't develop robust representations of *how events unfold over time* Clearest w number, but true in many cases, and it's part of why the quantity cases fail:

Yann LeCun @ylecun

@StanDehaene @GaryMarcus Actually, machines that are trained to deal with quantities do learn to deal with quantities. Gary merely says that machines trained to predict missing words don't learn much about quantities. Duh!

Yann LeCun battled me on every step of the journey, most notably in this rather bellicose and somewhat condescending “rear-guard battle” tweet, in which he said that all my concerns had long since been addressed

Yann LeCun @ylecun

@GaryMarcus Wrong. See this: arxiv.org/abs/1612.03969 Look at Table 2: Row 7 (counting) and 14 (time reasoning) both get 0 error rates (this is on bAbI tasks). When you are fighting a rear-guard battle, it's best to know when you adversary overtook your rear 3 years ago.

What’s changed since 2019?

I haven’t changed. I still often point out the dopey things that GPT-2’s grandchildren say. And I still attribute those problems to a lack of world models that represent the ways in which things unfold over time. I still see this as the central flaw in large language models.

And GPT hasn’t really changed, either. Its output is more fluent, and lots of individual mistakes have been addressed. But more than three years on, it’s still—even after a new update a few days ago focused around math—having lot of problems with easy word problems like this:

Ernest Davis @ErnestSDavis

I ran this just now, two days after the January 30 update with its huge improvement to ChatGPTs math ability:

and this

Ernest Davis @ErnestSDavis

To test replicability, I ran this simplified version with my typo corrected, and got this even worse result.

§

You know who has changed? Yann LeCun. LeCun’s not accusing me any more of fighting a rearguard battle, when it comes to large language models.

Instead, he’s up there with me (never citing me, of course), making the exact same points I was making in 2019, as if he never disagreed.

Yann LeCun @ylecun

Language abilities != Thinking. Or why LLMs such as ChatGPT can eloquently spew complete nonsense. Their grasp of reality is very superficial. theatlantic.com/technology/arc… This piece in the Atlantic comments on a paper by the MIT Cognitive Science crowd arxiv.org/abs/2301.06627

Oh, one other change, too: a bunch of other people have taken notice what LeCun is up to, and how he has retrenched, without a shred of acknowledgement, as if Oceania had always been at war with Eurasia. After the tweet above, one person DM’d me this morning, kidding, “Be honest. Did you hack his account to write that? lol”.

Another (someone I don’t know) wrote a long thread about LeCun’s constant gaslighting, which starts like his:

Alessandro Perilli 🇺🇦 @giano

The ultimate recipe for success for #AI/#ML researchers in 7 easy steps: Step 1 - Publicly praise and endorse an AI large language model released by your research team and designed to generate authoritative output while, in reality, it spits out absurd and/or dangerous nonsense.

My favorite is Step 6, “Never agree with the aforementioned critics but start mimicking their approach.”

To sum up, Yann LeCun may have changed, but I haven’t. My goalposts haven’t shifted; I still want A(G)I that knows what the f___ it’s talking about, and GPT still doesn’t.

Maybe things will be different in another three years.

Gary Marcus (@garymarcus), scientist, bestselling author, and entrepreneur, is a skeptic about current AI but genuinely wants to see the best AI possible for the world—and still holds a tiny bit of optimism. Sign up to his Substack (free!), and listen to him on Ezra Klein. His most recent book, co-authored with Ernest Davis, Rebooting AI, is one of Forbes’s 7 Must Read Books in AI.