Is Yann LeCun’s new company trying to do something similar to what Li Fei-Fei is working on?
If I remember correctly, I read an interview where LeCun argued that what current LLMs do is nothing like animal intelligence. He used the example of a dog: if a dog wants to jump to a higher place, it first estimates whether the jump is possible before acting. That kind of embodied prediction and world modeling is fundamentally different from next-token prediction.
Instead of shoveling random boxes of training data into the neural net, let’s shovel in specialized training data. If we pray really hard, something magical might come out.
All these “world model” outfits are the same, fundamentally flawed nonsense. For LeCun it’s generative video. For others it’s 3D models, or video game engines, or whatever.
More of the same intellectual laziness that got us into this mess.
'Our domestic pets including cats and dogs, are good sources for experimental observation of brain behavior. Cats do not compute heights or distances before they jump.
Dogs do not consider which food item to choose - a smaller one that might be filled,
versus an empty (hollow) one that looks bigger - they would likely choose the bigger one
because it ‘looks big’. Given two pairs of dots on paper, one pair with closer dots and
the other with farther ones, we do not need a ruler or calculator (to compute Euclidean
distance after imposing a coordinate system and assigning (x,y) coordinates to the four
dots!). What we see in case after case is simple estimation (which can indeed go wrong,
but shows that NSI is at work), or associative learning (for most dogs, hearing ‘ball’ in the
language spoken around them means play time). Estimation, association, imitation etc.
are plausible faculties that could have evolved in natural brains, all based on directness
(immediacy) rather than deliberate (including symbol-oriented) reasoning. '
I remember what Dr. Li Fei-Fei once said: imagine a house is on fire, but the robot is still calmly calculating the next move in a chess game, completely unaware of what’s happening around it.
I’m not an expert in this field, but I feel rather pessimistic about this direction. Right now, most of these systems are still fundamentally text based. The inputs are already enormous, and I can hardly imagine how much money, energy, and infrastructure it would take to build a truly global model that understands the world as it unfolds, not just as text on a screen.
Hi Xian, indeed. And, in the opp direction so to speak, animals (who are non-human-language-verbal) know by instinct and training, when there is danger to their human owners or themselves, and act accordingly.
My background is in CG, and materials science. So I can confidently say this - every generative "world" model is a cheap approximation of the real world, nothing more than glorified videogame levels. In any videogame, the "physics" is restricted to what the engine can (is programmed to) calculate. Same with these GenAI worlds. IOW it's for the most part a "look but can't touch" world. Eg the WM can generate a bush with pretty flowers, but I won't be able to walk up to one, pull its petals apart, rearrange them on the ground, then blow air to make them scatter, then step on them to make them leave a wet impression on the ground, then ball them up and stuff it in my jeans pocket. There are 1000s of things I can't do AFTER all this (to the poor flower!) but I'll spare the details.
Wittgenstein wrote: "The world is everything that is the case." Is a world model a description of every object and of all the actual or potential relations between every object? Do you plan to explicitly write down a list of the things you can or cannot put on a kitchen table (a spoon, a gallon of water without container, a 5 m wide cube of lead, a smile, a balloon filled with helium, etc.)? Things you can store in a cardboard box? Things you can do while driving a formula-one car?
That would be infeasible: the model would soon become bigger than the world itself. Some level of abstraction would be needed to simplify and structure your model in order to keep it manageable. How do you prevent an exponential explosion of facts and relations while building a world model? How do you know you didn't overlook something that makes your world model believe that you can put X on a kitchen table when this is impossible?
Every person has a world model, which is incomplete and full of mistakes, mostly built by trial and error, but good enough to survive most of the time. How long would a robot with its own artificial world model survive if we let it roam about? It may also have to learn by trial and error. Many would end up on a scrap heap after a mishap. There is simply too much to learn for it to be possible to be explicitly taught.
"Do you plan to explicitly write down a list of the things you can or cannot put on a kitchen table ... the model would soon become bigger than the world itself".
We can collect and write down knowledge about the world, and with recent AI efforts we do a lot that, multiplied by a few billion.
The issue is what you do with all that data. Do you put it in a neat logical framework, or do you keep it a loosely organized haystack? I agree that neat frameworks will choke. The best results we have achieved, after many decades of trying, is to try to respect the complexity and not organize it too rigidly.
That's quite the strawman that you have created and then knocked down. Wittgenstein is irrelevant ... sure the world is every fact about the world, but a "world model" is not a model of every fact about the world, as you yourself manage to realize a few words later.
> How do you prevent an exponential explosion of facts and relations while building a world model?
Why would there be? You seem to envision some process of building a world model that results in an "exponential explosion" ... well, don't do it that way.
> How do you know you didn't overlook something that makes your world model believe that you can put X on a kitchen table when this is impossible?
How do you know that there aren't bugs in your programs? You don't, of course ... but there are problem solving approaches that are less prone to bugs than others, and ways of building fault-tolerant systems.
You're putting the cart before the horse, inventing bogeymen before the engineering work has been done. You have no idea what world model building approaches people will come up with ... wait to see them first.
> mostly built by trial and error
That's your biggest mistake (among so many packed into a small number of words)--human world models are the result of a combination innate features honed by billions of years of evolution, communication with other humans, and a smattering of experimentation. Our world models are absolutely not "mostly built by trial and error"--a great deal of research shows otherwise. And the experimentation we do is not random or scattershot, it's goal-directed.
Funny that in accusing me of building a straw man you have built your own. Our internal world model was indeed shaped by evolution, which is by definition by trial and error. There is no such thing as goal-directed evolution. Back to biology class for you.
There are people I trust to be true to their beliefs. Gary Marcus and Yann Lecun are two of them.
You don't have to buy into everything they say, but you really ought to listen and understand what they're saying because somewhere in there are gems that provide insight into an incredible industry that has been saturated with opinions from others who are far less informed or knowledgeable?.
Many weekends while programming for a minicomputer company outside Boston, I walked past the Harvard Bookstore and admired a large yellow-gold book. Eventually I bought and read E. O. Wilson’s Insect Societies. A small ant changed my life. Pseudomyrmex ferruginea colonizes bullhorn acacia thorns. In exchange for nutrients, the ants defend it from predators. Years later, hiking in Guatemala, I saw an acacia tree covered with thorns with a hole in each. When I tapped it, ants swarmed out over my hand and around the tree. Wilson had written that on a warm day, winged queens emerge from a chrysalis and fly off, mate with a winged male, then each finds and burrows into an unoccupied acacia thorn and lays eggs to start a new colony. The ant that bit my finger painfully had a world model of herbivore acacia-eaters that was not taught or learned. I realized for the first time that children raised in a great environment won’t be great if some genetically programmed behaviors need to be addressed in each generation. I returned to university to study psychology and anthropology.
Software can’t become AGI by learning a world model like a baby does, because babies arrive with a world model developed over millions of years. Watching a troop of infant primates, you see that our world models overlap, without training or anything that can imaginably be made explicit. There is perceptual and cognitive intelligence, such as instinctive reactions to the shape of a snake that both human infants and primates show, and also social and emotional intelligences. Our intelligence was shaped to function in a social group that is co-dependent for finding food, avoiding predators, and raising the next generation. Jan Steen is right, there is too much to learn.
So, build an ant mind, if you can, and work up from there, as nature did (evolution didn't do "clean sheet" designs).The first step might be the hardest.
I glimpse old psychological and epistemlogical arguments through the computerese in the comments here,e.g. "tabla rasa"/"psychologically, all behavior is learned". It clearly ain't, and can't, be.
It’s nice of you to be humble about LeCun’s situation, but I still recall the Twitter exchanges he had with you regarding LLMs as though those were the answers to AGI, and for that I’ll never be able trust his view on AI matters regardless of his past experience and achievements. Yes, we all make mistakes, but he is arrogance was…how should I put it?…very French 🤣🙄
After reading your post on LeCun, I went back to your 2020 paper “The Next Decade in AI” (2002.06177), especially the sections arguing that robust AI requires explicit, updateable internal models of the world rather than purely implicit representations.
One thing that struck me is how much the debate hinges not just on learning world models, but on where their structured state actually lives. I’ve been working on a technology aimed at providing an external, shared context substrate - something models can read from and write to - so that learned dynamics and abstractions don’t have to carry all persistent world structure internally.
I’m not suggesting this solves the learning or reasoning challenges you raise, but reading both pieces together made me wonder whether separating learned dynamics from a persistent, corrigible world state could help address some of the brittleness and reliability issues you’ve highlighted.
LeCun's 2022 position paper laid out very clearly his vision for the future of AI. The fundamental question, as you suggest, is what representational schemas should be used to represent this actionable, physics-aware world knowledge.
The litmus test for Advanced Machine Intelligence, as envisaged by LeCun, is not Terence Tao level intelligence. It's something closer to housecat intelligence. Your 2020 paper seemed to focus on emulating human level intelligence, including language. Hence the suggestion that some kind of formal language (hybrided with some kind of action management system) is required to represent the cognitive model. LeCun hasn't responded to that suggestion. But if your goal is achieving housecat intelligence, it's hard to see how formal languages would be useful.
As you mention, Pearl's seminal work on Causal Inference offers a formal language for causality, but the question is, how can such representations be built up emergently, from direct interaction with the world, with no human operator in the loop to design them and test them?
I'm not sure JEPA is the right vehicle for AMI, even as LeCun imagines it. Even on LeCun's own telling, there are limitations, very hard to circumvent, in its' ability to generalise from experience and apply that knowledge intelligently. It's a hard problem. But I respect LeCun. At least he's looking at the right problems, even if the solutions aren't there yet.
Did anyone see The Information’s prediction that someone will invent AI that learns in real time in 2026? Seems wild based on my understanding of the field.
From its Wikipedia page: "the Financial Times had expressed interest "in a possible takeover," but also noted close ties to leaders in the tech industry, which publication covers."
Gary Marcy, that was a very interesting read. One thing you said about credit makes me sad and angry at myself.
At the same MIT review where Yann first wowed everyone by waving his new convolutional ASIC chip-and-camera around the room, another much quieter fellow was also there. The quieter fellow had done much of the same kind of foundational work, but had no chip and was getting no attention. I was impressed by his story, and I recall making some effort to get more attention and funding for him later.
What makes me angry and disappointed in myself is this: I cannot now recall the quieter researcher’s name.
Pure Tech Comedy gold … 🤪😉 … nice to know that LeCun stuck to his guns … and called it out and particularly amusing given Zuck’s heritage … competitive intensity is high …. I will never forget in Summer 2024 I was Chairing a Board meeting and a certain person - who will remain nameless said - “Have you seen the LLM league table ?…” complete silence until a Board member said “why should we care ?… “ 😆 and so it continues #dontstop sharing Gary - this made me LOL 😂 a lot
This article is overall rather generous to LeCun, all past things considering.
It must be stated, even though not touched upon in the article, that LeCun was consistently against LLM. Galactica was Zuck's product, which LeCun had to pay lip service to, as he lead the lab. He did not believe LLM would lead to AGI.
"LeCun talks about world models, but (because of what appears to be an ego-related mental block against classical symbolic AI"
This is not fair. LeCun genuinely believes world models must not be beholden to human-produced rules, but those must be discovered from data. Bringing here ego issues, where genuine scientific difference exists, is not helpful.
As to to JEPA vs LLM the world is big enough for both. LLM compress world knowledge better, but won't be enough likely for a robot in the world. We will see.
Ahhhh I do so love reading the latest goss from the world of AI, especially as written by you, Gary. I'm also very curious ... Prof Donald Hoffman (who wrote the Case Against Reality) has 'proven' that time and space aren't 'real' in the sense we assume they are and so cannot be the foundation of our scientific assumptions. If we include his work in a world model of AI, how would that impact the possibilities of getting a more useful artificial intelligence?
> time and space aren't 'real' in the sense we assume they are
Who is "we"?
> and so cannot be the foundation of our scientific assumptions.
Common naive notions about time and space are indeed not the foundations of modern science. As for "real"--who cares? It's models all the way down. Hoffman says we're suffering from an illusion--big deal; the math still works. Take a look at https://www.youtube.com/watch?v=nWJFtLScii4
> If we include his work in a world model of AI
I have no idea what this handwaving means.
Look, Hoffman is a philosopher (in this regard; he's also pegged as a "cognitive scientist", but this stuff isn't science). 99% of philosophers are hopelessly conceptually muddled, and this includes both Hoffman and his critics (https://www.reddit.com/r/consciousness/comments/14gdqeh/donald_hoffman_the_case_against_reality_this_book/). Philosophy is supposed to be about asking questions so as to clarify our understanding ... when philosophers start making assertions they are out of their lane. Their speculations certainly should not be "include[d] ... in a world model of AI" -- which is pretty much a meaningless jumble of words.
This will probably fall on deaf ears and blind eyes, but LeCun and most of the tech industry and experts' folly is trying to chase AGI and superintelligence. At this point, it's basically the equivalent of trying to find god - a fool's errand. China caught on quite soon and forsake this goose chase and are now ahead of everyone, not because they necessarily have better technology, but rather because they are finding practical and useful ways to apply it, however limited and flawed those uses still are.
Wild read. I was totally unaware that this happened.
Is Yann LeCun’s new company trying to do something similar to what Li Fei-Fei is working on?
If I remember correctly, I read an interview where LeCun argued that what current LLMs do is nothing like animal intelligence. He used the example of a dog: if a dog wants to jump to a higher place, it first estimates whether the jump is possible before acting. That kind of embodied prediction and world modeling is fundamentally different from next-token prediction.
In a nut/dog shell! Prediction is not all it's cracked (sorry) up to be.
Instead of shoveling random boxes of training data into the neural net, let’s shovel in specialized training data. If we pray really hard, something magical might come out.
All these “world model” outfits are the same, fundamentally flawed nonsense. For LeCun it’s generative video. For others it’s 3D models, or video game engines, or whatever.
More of the same intellectual laziness that got us into this mess.
Exactly.
If I may quote myself (from a paper), lol...
'Our domestic pets including cats and dogs, are good sources for experimental observation of brain behavior. Cats do not compute heights or distances before they jump.
Dogs do not consider which food item to choose - a smaller one that might be filled,
versus an empty (hollow) one that looks bigger - they would likely choose the bigger one
because it ‘looks big’. Given two pairs of dots on paper, one pair with closer dots and
the other with farther ones, we do not need a ruler or calculator (to compute Euclidean
distance after imposing a coordinate system and assigning (x,y) coordinates to the four
dots!). What we see in case after case is simple estimation (which can indeed go wrong,
but shows that NSI is at work), or associative learning (for most dogs, hearing ‘ball’ in the
language spoken around them means play time). Estimation, association, imitation etc.
are plausible faculties that could have evolved in natural brains, all based on directness
(immediacy) rather than deliberate (including symbol-oriented) reasoning. '
Lol, the dog doesn't calculate - crucial difference.
Babies don't scratch their non-existent beards and HYPOTHESIZE about the world, as absurdly claimed.
I remember what Dr. Li Fei-Fei once said: imagine a house is on fire, but the robot is still calmly calculating the next move in a chess game, completely unaware of what’s happening around it.
I’m not an expert in this field, but I feel rather pessimistic about this direction. Right now, most of these systems are still fundamentally text based. The inputs are already enormous, and I can hardly imagine how much money, energy, and infrastructure it would take to build a truly global model that understands the world as it unfolds, not just as text on a screen.
Hi Xian, indeed. And, in the opp direction so to speak, animals (who are non-human-language-verbal) know by instinct and training, when there is danger to their human owners or themselves, and act accordingly.
My background is in CG, and materials science. So I can confidently say this - every generative "world" model is a cheap approximation of the real world, nothing more than glorified videogame levels. In any videogame, the "physics" is restricted to what the engine can (is programmed to) calculate. Same with these GenAI worlds. IOW it's for the most part a "look but can't touch" world. Eg the WM can generate a bush with pretty flowers, but I won't be able to walk up to one, pull its petals apart, rearrange them on the ground, then blow air to make them scatter, then step on them to make them leave a wet impression on the ground, then ball them up and stuff it in my jeans pocket. There are 1000s of things I can't do AFTER all this (to the poor flower!) but I'll spare the details.
It's all a joke.
Glorified game engines, where collision for ex. is calculated using GJK (for ex) algorithm.
There is no "real" gravity, energy, time... there's just... scores of POLYGONS.
Wittgenstein wrote: "The world is everything that is the case." Is a world model a description of every object and of all the actual or potential relations between every object? Do you plan to explicitly write down a list of the things you can or cannot put on a kitchen table (a spoon, a gallon of water without container, a 5 m wide cube of lead, a smile, a balloon filled with helium, etc.)? Things you can store in a cardboard box? Things you can do while driving a formula-one car?
That would be infeasible: the model would soon become bigger than the world itself. Some level of abstraction would be needed to simplify and structure your model in order to keep it manageable. How do you prevent an exponential explosion of facts and relations while building a world model? How do you know you didn't overlook something that makes your world model believe that you can put X on a kitchen table when this is impossible?
Every person has a world model, which is incomplete and full of mistakes, mostly built by trial and error, but good enough to survive most of the time. How long would a robot with its own artificial world model survive if we let it roam about? It may also have to learn by trial and error. Many would end up on a scrap heap after a mishap. There is simply too much to learn for it to be possible to be explicitly taught.
"Do you plan to explicitly write down a list of the things you can or cannot put on a kitchen table ... the model would soon become bigger than the world itself".
We can collect and write down knowledge about the world, and with recent AI efforts we do a lot that, multiplied by a few billion.
The issue is what you do with all that data. Do you put it in a neat logical framework, or do you keep it a loosely organized haystack? I agree that neat frameworks will choke. The best results we have achieved, after many decades of trying, is to try to respect the complexity and not organize it too rigidly.
That's quite the strawman that you have created and then knocked down. Wittgenstein is irrelevant ... sure the world is every fact about the world, but a "world model" is not a model of every fact about the world, as you yourself manage to realize a few words later.
> How do you prevent an exponential explosion of facts and relations while building a world model?
Why would there be? You seem to envision some process of building a world model that results in an "exponential explosion" ... well, don't do it that way.
> How do you know you didn't overlook something that makes your world model believe that you can put X on a kitchen table when this is impossible?
How do you know that there aren't bugs in your programs? You don't, of course ... but there are problem solving approaches that are less prone to bugs than others, and ways of building fault-tolerant systems.
You're putting the cart before the horse, inventing bogeymen before the engineering work has been done. You have no idea what world model building approaches people will come up with ... wait to see them first.
> mostly built by trial and error
That's your biggest mistake (among so many packed into a small number of words)--human world models are the result of a combination innate features honed by billions of years of evolution, communication with other humans, and a smattering of experimentation. Our world models are absolutely not "mostly built by trial and error"--a great deal of research shows otherwise. And the experimentation we do is not random or scattershot, it's goal-directed.
Funny that in accusing me of building a straw man you have built your own. Our internal world model was indeed shaped by evolution, which is by definition by trial and error. There is no such thing as goal-directed evolution. Back to biology class for you.
You're conflating the evolutionary construction of our genome with the human individual's construction of an object oriented world model.
Not at all. It's evidently a combination of both.
WHAT is a combination of both? Do you even understand what "conflating" means?
Gawd, what an imbecile.
> Funny that in accusing me of building a straw man you have built your own.
Whataboutism is a sign of fundamental dishonesty.
> Our internal world model was indeed shaped by evolution, which is by definition by trial and error.
No, it certainly isn't defined that way.
> There is no such thing as goal-directed evolution.
Another strawman ... I said the experimentation we do, not what evolution does.
> Back to biology class for you.
Such projection ... claiming that evolution is by definition trial and error is profoundly ignorant.
Yes and no. Rather "some massive level of redundancy, with errors built INTO it.
There are people I trust to be true to their beliefs. Gary Marcus and Yann Lecun are two of them.
You don't have to buy into everything they say, but you really ought to listen and understand what they're saying because somewhere in there are gems that provide insight into an incredible industry that has been saturated with opinions from others who are far less informed or knowledgeable?.
Many weekends while programming for a minicomputer company outside Boston, I walked past the Harvard Bookstore and admired a large yellow-gold book. Eventually I bought and read E. O. Wilson’s Insect Societies. A small ant changed my life. Pseudomyrmex ferruginea colonizes bullhorn acacia thorns. In exchange for nutrients, the ants defend it from predators. Years later, hiking in Guatemala, I saw an acacia tree covered with thorns with a hole in each. When I tapped it, ants swarmed out over my hand and around the tree. Wilson had written that on a warm day, winged queens emerge from a chrysalis and fly off, mate with a winged male, then each finds and burrows into an unoccupied acacia thorn and lays eggs to start a new colony. The ant that bit my finger painfully had a world model of herbivore acacia-eaters that was not taught or learned. I realized for the first time that children raised in a great environment won’t be great if some genetically programmed behaviors need to be addressed in each generation. I returned to university to study psychology and anthropology.
Software can’t become AGI by learning a world model like a baby does, because babies arrive with a world model developed over millions of years. Watching a troop of infant primates, you see that our world models overlap, without training or anything that can imaginably be made explicit. There is perceptual and cognitive intelligence, such as instinctive reactions to the shape of a snake that both human infants and primates show, and also social and emotional intelligences. Our intelligence was shaped to function in a social group that is co-dependent for finding food, avoiding predators, and raising the next generation. Jan Steen is right, there is too much to learn.
So, build an ant mind, if you can, and work up from there, as nature did (evolution didn't do "clean sheet" designs).The first step might be the hardest.
I glimpse old psychological and epistemlogical arguments through the computerese in the comments here,e.g. "tabla rasa"/"psychologically, all behavior is learned". It clearly ain't, and can't, be.
It’s nice of you to be humble about LeCun’s situation, but I still recall the Twitter exchanges he had with you regarding LLMs as though those were the answers to AGI, and for that I’ll never be able trust his view on AI matters regardless of his past experience and achievements. Yes, we all make mistakes, but he is arrogance was…how should I put it?…very French 🤣🙄
After reading your post on LeCun, I went back to your 2020 paper “The Next Decade in AI” (2002.06177), especially the sections arguing that robust AI requires explicit, updateable internal models of the world rather than purely implicit representations.
One thing that struck me is how much the debate hinges not just on learning world models, but on where their structured state actually lives. I’ve been working on a technology aimed at providing an external, shared context substrate - something models can read from and write to - so that learned dynamics and abstractions don’t have to carry all persistent world structure internally.
I’m not suggesting this solves the learning or reasoning challenges you raise, but reading both pieces together made me wonder whether separating learned dynamics from a persistent, corrigible world state could help address some of the brittleness and reliability issues you’ve highlighted.
Has anyone got an 'evolution-friendly' model of the real world yet? That might be a good place to start.
Interesting discussion.
LeCun's 2022 position paper laid out very clearly his vision for the future of AI. The fundamental question, as you suggest, is what representational schemas should be used to represent this actionable, physics-aware world knowledge.
The litmus test for Advanced Machine Intelligence, as envisaged by LeCun, is not Terence Tao level intelligence. It's something closer to housecat intelligence. Your 2020 paper seemed to focus on emulating human level intelligence, including language. Hence the suggestion that some kind of formal language (hybrided with some kind of action management system) is required to represent the cognitive model. LeCun hasn't responded to that suggestion. But if your goal is achieving housecat intelligence, it's hard to see how formal languages would be useful.
As you mention, Pearl's seminal work on Causal Inference offers a formal language for causality, but the question is, how can such representations be built up emergently, from direct interaction with the world, with no human operator in the loop to design them and test them?
I'm not sure JEPA is the right vehicle for AMI, even as LeCun imagines it. Even on LeCun's own telling, there are limitations, very hard to circumvent, in its' ability to generalise from experience and apply that knowledge intelligently. It's a hard problem. But I respect LeCun. At least he's looking at the right problems, even if the solutions aren't there yet.
Did anyone see The Information’s prediction that someone will invent AI that learns in real time in 2026? Seems wild based on my understanding of the field.
You actually pay to read such rubbish?
From its Wikipedia page: "the Financial Times had expressed interest "in a possible takeover," but also noted close ties to leaders in the tech industry, which publication covers."
An excellent read, Gary M!
thank you
billionaires do that often - react like kids and put kids in charge over experience and wisdom
sadly it’s a pattern.
Well done Yann
Gary Marcy, that was a very interesting read. One thing you said about credit makes me sad and angry at myself.
At the same MIT review where Yann first wowed everyone by waving his new convolutional ASIC chip-and-camera around the room, another much quieter fellow was also there. The quieter fellow had done much of the same kind of foundational work, but had no chip and was getting no attention. I was impressed by his story, and I recall making some effort to get more attention and funding for him later.
What makes me angry and disappointed in myself is this: I cannot now recall the quieter researcher’s name.
There’s something oddly comforting about realizing that the egos of tech geniuses may be one of the few things promoting measured progress. 👀
Pure Tech Comedy gold … 🤪😉 … nice to know that LeCun stuck to his guns … and called it out and particularly amusing given Zuck’s heritage … competitive intensity is high …. I will never forget in Summer 2024 I was Chairing a Board meeting and a certain person - who will remain nameless said - “Have you seen the LLM league table ?…” complete silence until a Board member said “why should we care ?… “ 😆 and so it continues #dontstop sharing Gary - this made me LOL 😂 a lot
This article is overall rather generous to LeCun, all past things considering.
It must be stated, even though not touched upon in the article, that LeCun was consistently against LLM. Galactica was Zuck's product, which LeCun had to pay lip service to, as he lead the lab. He did not believe LLM would lead to AGI.
"LeCun talks about world models, but (because of what appears to be an ego-related mental block against classical symbolic AI"
This is not fair. LeCun genuinely believes world models must not be beholden to human-produced rules, but those must be discovered from data. Bringing here ego issues, where genuine scientific difference exists, is not helpful.
As to to JEPA vs LLM the world is big enough for both. LLM compress world knowledge better, but won't be enough likely for a robot in the world. We will see.
Ahhhh I do so love reading the latest goss from the world of AI, especially as written by you, Gary. I'm also very curious ... Prof Donald Hoffman (who wrote the Case Against Reality) has 'proven' that time and space aren't 'real' in the sense we assume they are and so cannot be the foundation of our scientific assumptions. If we include his work in a world model of AI, how would that impact the possibilities of getting a more useful artificial intelligence?
Or one that on its own just gives up entirely?
🤣🤣🤣
> time and space aren't 'real' in the sense we assume they are
Who is "we"?
> and so cannot be the foundation of our scientific assumptions.
Common naive notions about time and space are indeed not the foundations of modern science. As for "real"--who cares? It's models all the way down. Hoffman says we're suffering from an illusion--big deal; the math still works. Take a look at https://www.youtube.com/watch?v=nWJFtLScii4
> If we include his work in a world model of AI
I have no idea what this handwaving means.
Look, Hoffman is a philosopher (in this regard; he's also pegged as a "cognitive scientist", but this stuff isn't science). 99% of philosophers are hopelessly conceptually muddled, and this includes both Hoffman and his critics (https://www.reddit.com/r/consciousness/comments/14gdqeh/donald_hoffman_the_case_against_reality_this_book/). Philosophy is supposed to be about asking questions so as to clarify our understanding ... when philosophers start making assertions they are out of their lane. Their speculations certainly should not be "include[d] ... in a world model of AI" -- which is pretty much a meaningless jumble of words.
This will probably fall on deaf ears and blind eyes, but LeCun and most of the tech industry and experts' folly is trying to chase AGI and superintelligence. At this point, it's basically the equivalent of trying to find god - a fool's errand. China caught on quite soon and forsake this goose chase and are now ahead of everyone, not because they necessarily have better technology, but rather because they are finding practical and useful ways to apply it, however limited and flawed those uses still are.