The Road to AI We Can Trust

Share this post

Some things @GaryMarcus might say

garymarcus.substack.com

Some things @GaryMarcus might say

No, this article was not written by ChatGPT

Gary Marcus
Feb 4, 2023
39
6
Share
Share this post

Some things @GaryMarcus might say

garymarcus.substack.com

Most people think of Gary Marcus and Yann LeCun as matter and anti-matter (or perhaps the other way around). But that wasn’t always the case. One upon a time Gary Marcus and Yann LeCun were friends; they did a number of public events together, and even appeared side-by-side in a famous photo together, which LeCun posted on Facebook, at the time describing Gary Marcus as his favorite critic. (Later, immediately after LeCun won the Turing Award along with Hinton and Bengio, the photo circulated again, with Marcus cropped out).

Gary Marcus, Yann LeCun, Geoff Hinton, Andrew Ng, in Montreal 2014.

Indeed in 2017, Marcus and LeCun appeared together in a very civilized, and very interesting debate, moderated by David Chalmers at NYU. Marcus opened the debate with a list of things the two agreed up, and when Marcus earlier that day sent the list to LeCun, LeCun wrote back swiftly, and agreed on their points of agreement:

The dinner afterward, with Daniel Kahneman and others also present, was a good time for all.

Things changed, seemingly for good, on January 2, 2018, when Marcus posted his infamous Deep Learning: A Critical Appraisal, to which LeCun did not take kindly; LeCun tossed it off as “mostly wrong”, and the Twitter Wars began. Articles were written about these battles.

Sometimes the wars quieted down, only to rage again.

In 2021, the wars resumed

Many believed that these wars would never end; the disagreements seemed too intense. In November 2022, Galactica came, and the wars became even fiercer. Insults were hurled. Credentials were lied about. Lawsuits (for libel) were briefly (very briefly) contemplated. Things were bleak. Neither wished to speak to the other. The once friendly duo, who had amicably shared a stage several times in the mid 2010’s seemed irreconcilable.

And then, at the end of November … ChatGPT rolled in.

§

ChatGPT succeeded, where had Galactica (pulled after three controversial days) failed. All eyes turned to OpenAI. And well, LeCun began to channel Marcus.

Nearly every word below in this Twitter-style Ask Me Anything was (except as noted) written by Yann LeCun this morning, but may as well have been written by Marcus.

Twitter avatar for @ylecun
Yann LeCun @ylecun
On the highway towards Human-Level AI, Large Language Model is an off-ramp.
9:39 AM ∙ Feb 4, 2023
830Likes90Retweets
Twitter avatar for @hughhowey
Hugh Howey @hughhowey
@ylecun Human level? LLMs are already better than most humans on many tasks.
1:16 PM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@hughhowey No. They are better than many humans on a few tasks. But they make very stupid mistakes of common-sense that a 4 year-old, a chimp, a dog, or a cat would never make. LLMs have a more superficial understanding of the world than a house cat.
1:22 PM ∙ Feb 4, 2023
40Likes2Retweets
Twitter avatar for @FIQureshi1
Faisal Qureshi @FIQureshi1
@ylecun How about Image Gen Models (like #stablediffusion etc)
1:20 PM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@FIQureshi1 Also useful as a tool, with interesting underlying concepts (like diffusion models), but definitely not on the highway towards HLAI.
1:50 PM ∙ Feb 4, 2023
Twitter avatar for @OptimalBayes
Bayes-Optimal Agent @OptimalBayes
@ylecun Its not an off-ramp, but a landmark we see on the way
9:42 AM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@OptimalBayes It's a shiny casino you see off the highway. You can take the off-ramp, spend your money in the casino, and perhaps even win. But you risk forgetting why you were on the highway in the first place.
1:49 PM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@c7ddfc Transformers, like ConvNets and a few other architectural concepts, are clearly useful, both as tools towards HLAI and as components of practical applications. Getting machines to learn intuitive physics is an important but yet unsolved problem.
1:39 PM ∙ Feb 4, 2023
Twitter avatar for @ferdousbhai
0xferdous 🦇🔊 @ferdousbhai
@ylecun Yes, but still useful. LLM's existence doesn't stop people from pursuing AGI.
10:33 AM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@ferdousbhai Except for the people who believe human-level AI is just a matter of scaling up LLMs.
1:28 PM ∙ Feb 4, 2023
Twitter avatar for @ylecun
Yann LeCun @ylecun
@pcollellmir @lexfridman @ilyasut As I've said, Google and Meta have had ChatGPT-like things in their labs for quite a while. The only thing that the availability of ChatGPT has changed is that the general public now realizes what's possible. But the general public hasn't quite realized what's not possible yet.
1:27 PM ∙ Feb 4, 2023

Marcus would, I daresay, not disagree with a single word.

And it’s not just LLMs, hype, the limits of of scaling, and the need for better mechanisms for handling intuitive physics that Marcus and LeCun agree about. Marcus and LeCun would also agree that Cicero, which has received far less attention, deserves more, in part because of its capacity for planning:

Twitter avatar for @ylecun
Yann LeCun @ylecun
@Golisms One of the most interesting aspects of Cicero is its ability to plan. This ability to plan is a necessary component of autonomous intelligence and is completely absent from current LLMs.
1:52 PM ∙ Feb 4, 2023

Small wonder that Sam Altman stopped following Yann LeCun a few days ago.

§

Coda: I can neither confirm nor deny the truth of this speculation:

Twitter avatar for @alexjc
Alex J. Champandard ❄️ @alexjc@creative.ai @alexjc
I think @GaryMarcus figured out the password to this account and is now trolling people... (P.S. Congrats on implicitly winning the argument!)
Twitter avatar for @ylecun
Yann LeCun @ylecun
On the highway towards Human-Level AI, Large Language Model is an off-ramp.
9:58 AM ∙ Feb 4, 2023
17Likes3Retweets

But I can say that my work here is done.

Twitter avatar for @CasonCherry
Cason Cherry @CasonCherry
@GaryMarcus He’s coming’ home Gary.
Happy Season 3 GIF by Friends
4:52 PM ∙ Feb 3, 2023

Rumor has it that Marcus is ready to bury hatchet if LeCun is.

Share

39
6
Share
Share this post

Some things @GaryMarcus might say

garymarcus.substack.com
6 Comments
Rebel Science
Writes Rebel Science
Feb 4·edited Feb 4Liked by Gary Marcus

Haha. This was a pleasant and hilarious read. LeCun has come around and may even retrace his steps at some point and start over, although I doubt it. Reading the exchange on Twitter, I busted out laughing when I saw this tweet by an anonymous poster named Ubongo: "You are on an offramp." To which LeCun answered: "People have been telling me this during my entire career."

What's funny is that I agree with Ubongo. On the highway to AGI, LeCun took the offramp long ago when he made objective function optimization the core of his AI career. Deep learning is irrelevant to AGI. :-D

Expand full comment
Reply
suko
Writes Suko's Substack
Feb 4Liked by Gary Marcus

Ah I'm late to this very entertaining feud which I've been seeing on Twitter. Thanks for the background information. :)

You're both great. I wish you both well.

Expand full comment
Reply
4 more comments…
Top
New
Community

No posts

Ready for more?

© 2023 Gary Marcus
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing