I have to agree with those pointing out that a bunch of people jumping ship says more about the failure of LLM’s than it does about current approaches with world models. The idea that somewhere someone has conceptualized the model that will prove to be the gateway to AGI is nearly as presumptuous as claiming that merely scaling LLM’s will lead to general intelligence
Yes, I wonder how their EBM differs from previous heuristics/search approaches, and how they think it will fare outside of strict logical domains like math (where they did something lik e76% on a benchmark). Very little info so far, extraordinary claims. We will see.
Well, then hopefully there will be a REAL solution.
The current Rube-Goldberg-esque constructions of LLMs with syntax checkers, Unix tools (grep, sed, awk, …), Python runtimes, or whatever, glued onto sure have considerably improved reliability.
But I kind of liked the old-school pure LLM approach because at least everyone understood how fickle this was (except true believers).
Yes, it is more useful now. Still, bizarre logical bugs regularly slip through that cannot be caught by tools that essentially operate on a syntax level.
As long as this remains so, I think those tools should come with a warning, and it should be more obvious than this cute “This tool can make mistakes.” More like: “This tool will make mistakes that humans don't make.”
All roads lead to an AGI that mimics the core function of the mammalian neocortex AND it's recent add-on called language. Symbolic reasoning alone won't be AGI.
Yes indeed..the boys and girls at numenta.org are pretty close to uncovering that. They have a theoretical framework how the neocortex stores objects/concepts in 3D space and how these relate to one and other. Very convincing. The added functionality of language gives rise to new features but not needed for consciousness or intelligence, as per people who lose and later regain the language skill
Another possible interpretation - not saying it's the right one, just pointing out the possibility - is that for someone as consistently suspect as LeCun to get behind something counts as a strike against it. It's not a vindication of an approach for it to be believed in - it's a vindication if it's shown to work. Let's see.
This is true but I bet anyone would be the same if they were shat on and thrown under the bus for years whilst the very people who did it now jump on the bandwagon and pretend they never spoke against it.
I’m sure a simple acknowledgment of being wrong and acknowledging Marcus would go a long way to reducing these “I told you so” posts.
Lex Ovi: You are right, of course. It was Socrates who thanked those who corrected him (when he was indeed incorrect). But in the wrong company, it remains painful to advertise.
Fascinating inflection point in AI research. The timing is intresting because it coincides with the broader realization that pure scaling isn't delivering reasoning. I've been watching similiar pivots in enterprise AI where teams are quietly adding symbolic layers after hitting walls with pure neural approaches. The irony is that this validates what folks have been arguing for years, but it took the hype cylce peaking and then stalling for the industry to admit it.
I’m skeptical about the viability of a global model at current scales. The capital, energy, and infrastructure required are enormous, and it’s difficult to see how much more input would be needed to get there sustainably.
Absolutely! One can envision specific, narrow world models tailored to specific applications but to have one that is general at a non-trivial level would entail the assimilation of a tremendous amount of knowledge in addition to the rules underpinning the various disciplines. It would need to be amenable to constant updating and be capable of radical revision in response to revolutions such as, for instances, general relativity, quantum mechanics, plate tectonics, gene splicing, ...
In short I am skeptical about the willingness of the VCs and the commercial entities to make the investments in time and money to achieve it.
The market in automation is worth many trillions per year. I think we are still in the joyride phase. We will likely see ruthless focus on efficiency and on distilling the models as much as possible for any one job.
At some point LLM will be a thin decision-making layer, with the logic outsourced to specialized efficient tools as soon as it is unambiguous enough.
Couldn't it work if, rather than pursuing one AGI world model, researchers instead created smaller, specific models for different fields, i.e., a world model for medicine, another for engineering, etc.?). Such an approach would also require far less compute and energy and water resources.
Hi Gary - one thing: could I kindly request you refrain from saying, “LLMs are good at some things - for example coding” - which I’ve heard you say several times, most recently on your podcast with Steve Eisman. You are right to point out LLMs are deficient at performing human-level tasks; I want to emphasise that coding is not some kind of exception to this (why would this be the case - coding is *more difficult* than most tasks humans do!)
Coding is actually engineering. That means it’s a representation of a fundamental model, which is usually multi-dimensional. It is not a flat piece of text. Code lines not only represent data structures, they represent how those data structures are manipulated. Do you think for a second that an LLM “understands” the data structures being manipulated - and what is happening in the space of that data as those manipulations are happening - when it prints out flat sequences of code lines? Of course not, it’s pattern matching, as it does with any other task.
Consider this: great software systems actually mirror reality. I’ve come to understand it’s important to model a system on the real world, and not approximate it because it’s convenient. Here’s an example: a system always seemed to have machines at specific IP addresses on a network. Some people couldn’t understand why I chose to model “machines” separately from “IP addresses”. That’s more confusing! A machine always has an IP address! So why bother? Turns out we later needed “slave machines” - each connected to a parent machine, but without an IP on the network. No problem with my model, because machines and IP addresses are not modelled as being the same thing. How did I know to do this? Because machines and IP addresses aren’t the same thing in reality! Pretty simple, really.
Simple, but impossible for an LLM, because it doesn’t understand… drum roll… “world models”! You’re big on those, I think?
Coding is not the “best application” for LLMs, it’s the field most in peril. It would be great if you could refrain from perpetuating that.
(Anyone: bear in mind before replying: I didn’t even get started on this…)
I think you are completely correct. At the same time, it is important to point out that some LLMs can be helpful in assisting programmers and save them time. At least, I have heard some reputable programmers say this. And, for a low level, or at least new programmer like myself, I am finding them useful not only to learn the basics of a new program, but also to program in that language.
The question is not so much as to whether symbolic reasoning is needed, but as to how to put it in the mix.
With recent coding agents it is becoming rather clear that emulating symbolic reasoning works better than actually using a rigorous reasoning engine. Even though the object is code.
We are learning the same lesson for math theorem provers, which are making very nice progress. So, the the reasoning is kept fuzzy as much as possible as that allows for increased imagination and flexibility, and more formal methods are brought in later, for verification.
I often wonder how I solved some of these very problems, but, from an entirely different perspective. Indeed, I was trying to automate film making, first by conceptualizing a democratic cinema, then by knowing it would be "grown" rather than told (Chaos Theory) then by adding interactivity (ION), later by understanding metadata, personalization and crowdsourcing (Gracenote) and then later, by contextualizing the data (who, what, when, where, and how) coining hyper-personalization - content that changes based upon who you are (Sceneplay/Entertainment AI). It was reading Pearl's Book of Why where I first saw the shadow of someone who thought as I did, but from a completely different field. Ironically, I was neighbors with a world renowned neurologist when I founded Sceneplay -- and was reading his text books at the time. Neurosymbolic in a real sense, is how our brains process information -- with branching story structures (or in Pearl's approach, counterfactuals.) I was amazed that I ended up with a similar solution to a DO operator, simply through my creative work -- "come be a doer, don't be a viewer" I put on the packaging of the Bowie disc, I produced, and published, built using my Interactive Cinema System (type 1 non-branching, type 2 branching.) I know they aren't perfectly analogous, but similar enough that everyone is finally understanding that the narrative layer lives both inside and outside of us all.
Anne Greenberg: If I read you rightly, you (and a few others here) have undergone the breakthrough that is substantially metaphysical and ontological--very briefly, we are not "outsiders" of being but rather "insiders" to it--intelligent insiders. That's a huge insight in today's field of ideas and highly relevant to this discussion where many still view oneself as somehow sitting outside of being viewing reality (real being) it from some weird idea of a perch of some sort--<--there's the common metaphysical illusion that puts a stop to everything else that might put a throughline to how the world works (and a model).
However, there is the empirical/cognitive aspect to it that has to match it, and that is what is missing in most thinker/scientists today. The problems on both the ontological and the cognitional level are enormous (and enormously philosophical), and some are buried as immediate responders in our subconscious domain requiring even further massive changes (breakthroughs) that have not occurred yet in most (that I can tell in my eclectic search of such things).
But apparently (from my take on it) your statement is related in more common and artistic terms, but it has a direct relationship to a theoretical "model" to be found in Bernard Lonergan's work on Insight: A Study of Human Understanding--he wrote in the last century, but he has several "Collections" that cover the theoretical work which, for ontology/metaphysics and methodology, can be identified in your own note (and a few others') here. You should hang onto that set of insights. Catherine Blanche King
Thanks for your reply; I will try to see Lonergan's work, when time permits. I wrote about some of this in 2018 in a a series called Human Centered Storytelling…
Are you familiar was Oceanit, based in Hawaii, Gary? It was founded by a deeply Chomskyan friend of mine (another supervisee of Ian Roberts), but I know very little of their work. There are guaranteed not to be closet behaviourists though. Perhaps you know their work. Also, I was wondering if there was any work on the role of metaphor in potential approaches to AI? Or any state of the art papers on 'World Models'? It sounds Demis Hassibis has introduced elements of how humans learn into his work, and reaped rewards accordingly, but without getting anywhere near the sci-fi 'AGI in 5 years' stuff he peddles (he has a little touch of Musk about him, albeit while actually being a genius). Interested to hear what you think. All power to your elbow!
This makes me think of the detail discussed about AI in Pedro Domingos book, The Master Algorithm. He posed it as a quest in which various approaches would be taken over time.
I have to agree with those pointing out that a bunch of people jumping ship says more about the failure of LLM’s than it does about current approaches with world models. The idea that somewhere someone has conceptualized the model that will prove to be the gateway to AGI is nearly as presumptuous as claiming that merely scaling LLM’s will lead to general intelligence
Yes, I wonder how their EBM differs from previous heuristics/search approaches, and how they think it will fare outside of strict logical domains like math (where they did something lik e76% on a benchmark). Very little info so far, extraordinary claims. We will see.
Everybody's on their own AGI learning curve - and all roads ultimately lead to neurosymbolic AGI.
🎯
Well, then hopefully there will be a REAL solution.
The current Rube-Goldberg-esque constructions of LLMs with syntax checkers, Unix tools (grep, sed, awk, …), Python runtimes, or whatever, glued onto sure have considerably improved reliability.
But I kind of liked the old-school pure LLM approach because at least everyone understood how fickle this was (except true believers).
Yes, it is more useful now. Still, bizarre logical bugs regularly slip through that cannot be caught by tools that essentially operate on a syntax level.
As long as this remains so, I think those tools should come with a warning, and it should be more obvious than this cute “This tool can make mistakes.” More like: “This tool will make mistakes that humans don't make.”
Maybe not. Who knows is we ever reach AGI?
The world is too big for any one approach to work. For language and symbolic reasoning LLM hit the sweet spot.
It would be ridiculous for an LLM to reason in real-time about how to catch a falling cup, for example. Here need different kind of intelligence.
We are building a machine with many moving parts.
Yup. Catching a ball does not require symbolic reasoning but some training on reality and balls and ones body. Mostly a 3D plus T world model
I agree, Aaron, but the hubris of people who don't *want* to learn continues to impede progress for everyone...
All roads lead to an AGI that mimics the core function of the mammalian neocortex AND it's recent add-on called language. Symbolic reasoning alone won't be AGI.
'mimics' does a lot of heavy lifting here...
Yes indeed..the boys and girls at numenta.org are pretty close to uncovering that. They have a theoretical framework how the neocortex stores objects/concepts in 3D space and how these relate to one and other. Very convincing. The added functionality of language gives rise to new features but not needed for consciousness or intelligence, as per people who lose and later regain the language skill
Interesting. It would be interesting to read scientific papers about their approach
Here you go
https://www.numenta.com/resources/research-publications/papers/
In his book 'a thousand brains' Jeff Hawkins does a populist walkthrough. Highly recommended.
I don't believe Gary ever said that.
Another possible interpretation - not saying it's the right one, just pointing out the possibility - is that for someone as consistently suspect as LeCun to get behind something counts as a strike against it. It's not a vindication of an approach for it to be believed in - it's a vindication if it's shown to work. Let's see.
“I was right. I told you so.” = every Gary Marcus post
This is true but I bet anyone would be the same if they were shat on and thrown under the bus for years whilst the very people who did it now jump on the bandwagon and pretend they never spoke against it.
I’m sure a simple acknowledgment of being wrong and acknowledging Marcus would go a long way to reducing these “I told you so” posts.
it would 🤷♂️
as would news that was on some other axis besides what i anticipated.
but the news is what it is…
as would news that was on some other axis besides what i anticipated.
but the news is what it is…
Everyone has a right to change their opinion. No one has a right to pretend that they never held the opposing position before.
Lex Ovi: You are right, of course. It was Socrates who thanked those who corrected him (when he was indeed incorrect). But in the wrong company, it remains painful to advertise.
And no one with high social intelligence would fail to acknowledge those who anticipated and predicted their mistaken logic.
Fascinating inflection point in AI research. The timing is intresting because it coincides with the broader realization that pure scaling isn't delivering reasoning. I've been watching similiar pivots in enterprise AI where teams are quietly adding symbolic layers after hitting walls with pure neural approaches. The irony is that this validates what folks have been arguing for years, but it took the hype cylce peaking and then stalling for the industry to admit it.
I’m skeptical about the viability of a global model at current scales. The capital, energy, and infrastructure required are enormous, and it’s difficult to see how much more input would be needed to get there sustainably.
Absolutely! One can envision specific, narrow world models tailored to specific applications but to have one that is general at a non-trivial level would entail the assimilation of a tremendous amount of knowledge in addition to the rules underpinning the various disciplines. It would need to be amenable to constant updating and be capable of radical revision in response to revolutions such as, for instances, general relativity, quantum mechanics, plate tectonics, gene splicing, ...
In short I am skeptical about the willingness of the VCs and the commercial entities to make the investments in time and money to achieve it.
The market in automation is worth many trillions per year. I think we are still in the joyride phase. We will likely see ruthless focus on efficiency and on distilling the models as much as possible for any one job.
At some point LLM will be a thin decision-making layer, with the logic outsourced to specialized efficient tools as soon as it is unambiguous enough.
Couldn't it work if, rather than pursuing one AGI world model, researchers instead created smaller, specific models for different fields, i.e., a world model for medicine, another for engineering, etc.?). Such an approach would also require far less compute and energy and water resources.
Hi Gary - one thing: could I kindly request you refrain from saying, “LLMs are good at some things - for example coding” - which I’ve heard you say several times, most recently on your podcast with Steve Eisman. You are right to point out LLMs are deficient at performing human-level tasks; I want to emphasise that coding is not some kind of exception to this (why would this be the case - coding is *more difficult* than most tasks humans do!)
Coding is actually engineering. That means it’s a representation of a fundamental model, which is usually multi-dimensional. It is not a flat piece of text. Code lines not only represent data structures, they represent how those data structures are manipulated. Do you think for a second that an LLM “understands” the data structures being manipulated - and what is happening in the space of that data as those manipulations are happening - when it prints out flat sequences of code lines? Of course not, it’s pattern matching, as it does with any other task.
Consider this: great software systems actually mirror reality. I’ve come to understand it’s important to model a system on the real world, and not approximate it because it’s convenient. Here’s an example: a system always seemed to have machines at specific IP addresses on a network. Some people couldn’t understand why I chose to model “machines” separately from “IP addresses”. That’s more confusing! A machine always has an IP address! So why bother? Turns out we later needed “slave machines” - each connected to a parent machine, but without an IP on the network. No problem with my model, because machines and IP addresses are not modelled as being the same thing. How did I know to do this? Because machines and IP addresses aren’t the same thing in reality! Pretty simple, really.
Simple, but impossible for an LLM, because it doesn’t understand… drum roll… “world models”! You’re big on those, I think?
Coding is not the “best application” for LLMs, it’s the field most in peril. It would be great if you could refrain from perpetuating that.
(Anyone: bear in mind before replying: I didn’t even get started on this…)
I think you are completely correct. At the same time, it is important to point out that some LLMs can be helpful in assisting programmers and save them time. At least, I have heard some reputable programmers say this. And, for a low level, or at least new programmer like myself, I am finding them useful not only to learn the basics of a new program, but also to program in that language.
A man and a woman can make an intelligence, but they do it in the bedroom, not the laboratory.
Well, maybe in the laboratory, or bent over the sink, ... You get the idea.
The question is not so much as to whether symbolic reasoning is needed, but as to how to put it in the mix.
With recent coding agents it is becoming rather clear that emulating symbolic reasoning works better than actually using a rigorous reasoning engine. Even though the object is code.
We are learning the same lesson for math theorem provers, which are making very nice progress. So, the the reasoning is kept fuzzy as much as possible as that allows for increased imagination and flexibility, and more formal methods are brought in later, for verification.
I often wonder how I solved some of these very problems, but, from an entirely different perspective. Indeed, I was trying to automate film making, first by conceptualizing a democratic cinema, then by knowing it would be "grown" rather than told (Chaos Theory) then by adding interactivity (ION), later by understanding metadata, personalization and crowdsourcing (Gracenote) and then later, by contextualizing the data (who, what, when, where, and how) coining hyper-personalization - content that changes based upon who you are (Sceneplay/Entertainment AI). It was reading Pearl's Book of Why where I first saw the shadow of someone who thought as I did, but from a completely different field. Ironically, I was neighbors with a world renowned neurologist when I founded Sceneplay -- and was reading his text books at the time. Neurosymbolic in a real sense, is how our brains process information -- with branching story structures (or in Pearl's approach, counterfactuals.) I was amazed that I ended up with a similar solution to a DO operator, simply through my creative work -- "come be a doer, don't be a viewer" I put on the packaging of the Bowie disc, I produced, and published, built using my Interactive Cinema System (type 1 non-branching, type 2 branching.) I know they aren't perfectly analogous, but similar enough that everyone is finally understanding that the narrative layer lives both inside and outside of us all.
Anne Greenberg: If I read you rightly, you (and a few others here) have undergone the breakthrough that is substantially metaphysical and ontological--very briefly, we are not "outsiders" of being but rather "insiders" to it--intelligent insiders. That's a huge insight in today's field of ideas and highly relevant to this discussion where many still view oneself as somehow sitting outside of being viewing reality (real being) it from some weird idea of a perch of some sort--<--there's the common metaphysical illusion that puts a stop to everything else that might put a throughline to how the world works (and a model).
However, there is the empirical/cognitive aspect to it that has to match it, and that is what is missing in most thinker/scientists today. The problems on both the ontological and the cognitional level are enormous (and enormously philosophical), and some are buried as immediate responders in our subconscious domain requiring even further massive changes (breakthroughs) that have not occurred yet in most (that I can tell in my eclectic search of such things).
But apparently (from my take on it) your statement is related in more common and artistic terms, but it has a direct relationship to a theoretical "model" to be found in Bernard Lonergan's work on Insight: A Study of Human Understanding--he wrote in the last century, but he has several "Collections" that cover the theoretical work which, for ontology/metaphysics and methodology, can be identified in your own note (and a few others') here. You should hang onto that set of insights. Catherine Blanche King
Thanks for your reply; I will try to see Lonergan's work, when time permits. I wrote about some of this in 2018 in a a series called Human Centered Storytelling…
Humans see what they expect to…
That's cool and all but I can't help but wonder if the best outcome for any AI tech is failure.
Doug Leant must be smiling down on us ....
Prof. Marcus is the Arne Saknussemm of AI's 'Journey to the Center of the Earth'.
System 2 thinking might be solved by neursymbolic approaches. OLD school combined with new school Hybrid architecture
Are you familiar was Oceanit, based in Hawaii, Gary? It was founded by a deeply Chomskyan friend of mine (another supervisee of Ian Roberts), but I know very little of their work. There are guaranteed not to be closet behaviourists though. Perhaps you know their work. Also, I was wondering if there was any work on the role of metaphor in potential approaches to AI? Or any state of the art papers on 'World Models'? It sounds Demis Hassibis has introduced elements of how humans learn into his work, and reaped rewards accordingly, but without getting anywhere near the sci-fi 'AGI in 5 years' stuff he peddles (he has a little touch of Musk about him, albeit while actually being a genius). Interested to hear what you think. All power to your elbow!
This makes me think of the detail discussed about AI in Pedro Domingos book, The Master Algorithm. He posed it as a quest in which various approaches would be taken over time.