"describe the ways in which large language models (LLMs) can carry out tasks for which they were not specifically trained" - how do they know the LLMs were not specifically trained, have they examined the terabytes of training data or the millions if not billions of instances of RLHF to be able to claim that. To declare that LLMs can do that, the first step would be for the LLM to learn simple arithmetic and demonstrate it with big numbers with a lot of digits (that can not be simply remembered from the training data). Until an LLM can be demonstrated to be able to do that, all claims of a magically emergent AGI are just bla, bla, bla. So, count me among the confused too :) Also, I think 10 years from now people will look back at the current events and claims from prominent leaders in the AI field and just shake their heads in bemused disbelief.
Hi Gary. I basically agree with everything you said. I read the article a few days ago and was rather taken aback, especially by the condescending tone. Given the authors' stature in the field, they should know better than to make the kinds of pronouncements they did about today's systems. Along with Hinton's 60 minutes interview, there seems to be a lot of wishful thinking going around these days. This is shades of the '70s and '80s.
When I had started to read this blog, I believed that AGI might come soon, just behind the next corner. I have learned a lot since then and I don’t think so anymore. From the comments published here I tried to figure out how the LLMs were operating and I found a correspondence which is familiar to me as a physicist and engineer and which is the “black box model”. The “black box model” is a fully empirical model as opposed to knowledge based (laws of mechanics, thermodynamics, etc.) models. The empirical models are made of correlations based on experiments’ results. In their engineering form they are a set of polynomial equations relating outputs to input parameters, with a multitude of coefficients obtained by mathematical fitting the model output to some experimental data. There is no internal logic, schemes or rules, the model represents blindly the data it was fitted on. And it works well in mechanical or chemical engineering applications if a single well delimited phenomenon, or process is represented. It can be efficient for prediction if the user does not cross (even by the smallest amount) the value domain of parameters considered and does not try to represent a situation where an additional parameter is needed (even just a single parameter more). The LLMs have basically the same limitations. But the ambition with LLMs is quite extravagant as they will pretend to describe the entire world. If one wants to describe the entire world with a “black box model”, one would need an infinite number of parameters and an infinite number of coefficients. That is not possible. That’s why obviously LLMs are not very close to AGI. In order to get AGI, or just a reliable general representation of the world complexity, we will need a hybrid approach, a combination of empirical correlations, knowledge based equations and internal strong inferring rules.
Yours and Gary's well-reasoned and factual opposition against hype and nonsense is failing to shake existing convictions. Exactly the same happened in the 1960s, with Life stating in a famous article in 1970 that 'electronic persons' had arrived and Minsky in that article quoted saying that AGI would be there 'within 3 to 8 years'. Minsky then. Hinton c.s. now. History doesn't repeat, but it definitely rhymes. And the voices of realism are lost in a chorus of unfounded convictions. In the discussion on artificial intelligence, the role of human intelligence and how convictions form and are resistant to argument is the real interesting subject.
Oct 18, 2023·edited Oct 18, 2023Liked by Gary Marcus
You two gentlemen are way too polite. Agüera y Arcas and Norvig are not just wrong. They are miserably wrong. There is nothing in generative AI or LLMs that is even remotely useful to solving AGI. To crack AGI, the mainstream paradigm must not merely shift. It must be discarded entirely. A completely new model must arise to replace the current one. Unfortunately, I don't see this revolution coming from the deep learning generation.
PS. In my opinion, generalization and precise timing are key components of AGI.
Seems to me this was written while on auto-pilot, the temperature was a bit lower than Andreesen when he did his techno-optimist manifest, but still pretty much autopilot. You do that when you've given up on thinking for some reason or another. Maybe it's too hard, or too painful. These people just want to party down in the Good Place and leave all the worries to, well, whomever.
A more than welcome rebuttal. What I don’t get it why Arcad and Norvig would go as far as making the argument in the first place, when there is such clear evidence of the contrary...
LLMs are in the strongly rising part of the hype cycle. It will pass, either as generative AI hits clear limits or because a new hype cycle resulting from a new approach takes over.
The terminology is kinda confusing. LLMs are a (somewhat-)general technique in AI, in that they're able to do many kinds of tasks without being specifically trained on that task, e.g. few-shot learning. But that's not what capital-G General AI means.
Anecdotally, I've heard of fine-tuning an LLM on training materials a human might use (e.g. a spec or guide) and the LLM getting better at the task. That's maybe the first time I've seen something in "AI" that we might call "learning" if done by humans. Most older supervised learning tasks (even using deep learning) would seem bizarrely rote to humans. Like a human who needed to see thousands of pictures of a cat to be able to identify a cat would be considered to have some kind of severe learning disability. But it's a big leap from "maybe not completely unintelligent" to GAI.
The success of computational, ungrounded large language models based on the distributional hypothesis tells us a great deal about about how natural languages work but probably rather little about AGI other than to confirm that, contrary to Turing's imitation game, linguistic competency (however defined) is not a good measure of general intelligence.
The observation that linguistic competency can be programmed into a box, just as numeric competency can be programmed into a calculator, challenges our view of ourselves, our understanding of what makes humans intelligent, but this isn't relevant to AGI other than perhaps gently to question what we mean by "general intelligence", as discussed in the article here.
Humans use both numeric calculations and natural language, among other techniques, to reason about the world. Machines can be programmed to do the same but that doesn't make a system like GPT-4 any more intelligent than an electronic calculator, although both can be extremely useful.
Gary, I wonder how many times we can repeat these basic weaknesses to persons unwilling to acknowledge them. Part of me thinks we are encountering something akin to a culture war within AI circles. Why not? America breeds binaries that repel all reason or logic. There is a ideological impetus undergirding the constant redefinition of AGI on the part of the corporate tech giants. What can we do? Keep our heads. Realize larger forces at work behind the debate. And focus on questions of implementation.
Gary, I love your take on this. We as a community should build a solid AGI benchmark. Or even a subset of various general intelligences benchmark (physical intelligence, verbal intelligence, game solving, mathematics e.t.c).
Right now AGI is tossed around to get that sweet sweet VC handouts. But if there is a common agreed upon benchmark, then it's easy to say we're 10% hitting AGI benchmark but failing this domains.
Unless there is objective measure, most people will talk past each other.
A good summary of the state of the art. It’s amazing how much is changed and how little has changed. Like the philosopher Dreyfus said back in 1972 and revisited in the new edition of his book (1992) “What Computers *Still* Can’t Do". 😆
This not all Just getting pretty stupid; it’s been stupid since Turing laughed at the idea. And suppose we use plasma? And get it all together? This is the most patently recognized hoax in human history. Go and write program, code it good. Or take an axe and split some wood. This is all a tautology. It is a farce. I cannot believe it. What a lament for Feyerabend: he would have been sickened. Which is not the point though. We we are wit a hoax that a hoax because it is a tautology. Bayesian LLM or any Bayesian machine needs a Constantly infinite and dynamically changing with layer of knowledge about the depth of an atom,
and the depth of the depth, And on and on as quantum mechanics advances. And, that advance is the same as the advance as taking an axe to split wood.
Take the money away from these squonks at google and IBM and distribute it to the poor and to the war-stricken and just return to sanity. “AIG”: words only, and an acronym because people like these things before they become cliches. Which AIG will. Assholes In Google? “AIG”? Fools. a hoax. Infinitely worse than Cargo Cult Science. A lament for Feynman please.
"describe the ways in which large language models (LLMs) can carry out tasks for which they were not specifically trained" - how do they know the LLMs were not specifically trained, have they examined the terabytes of training data or the millions if not billions of instances of RLHF to be able to claim that. To declare that LLMs can do that, the first step would be for the LLM to learn simple arithmetic and demonstrate it with big numbers with a lot of digits (that can not be simply remembered from the training data). Until an LLM can be demonstrated to be able to do that, all claims of a magically emergent AGI are just bla, bla, bla. So, count me among the confused too :) Also, I think 10 years from now people will look back at the current events and claims from prominent leaders in the AI field and just shake their heads in bemused disbelief.
Hi Gary. I basically agree with everything you said. I read the article a few days ago and was rather taken aback, especially by the condescending tone. Given the authors' stature in the field, they should know better than to make the kinds of pronouncements they did about today's systems. Along with Hinton's 60 minutes interview, there seems to be a lot of wishful thinking going around these days. This is shades of the '70s and '80s.
When I had started to read this blog, I believed that AGI might come soon, just behind the next corner. I have learned a lot since then and I don’t think so anymore. From the comments published here I tried to figure out how the LLMs were operating and I found a correspondence which is familiar to me as a physicist and engineer and which is the “black box model”. The “black box model” is a fully empirical model as opposed to knowledge based (laws of mechanics, thermodynamics, etc.) models. The empirical models are made of correlations based on experiments’ results. In their engineering form they are a set of polynomial equations relating outputs to input parameters, with a multitude of coefficients obtained by mathematical fitting the model output to some experimental data. There is no internal logic, schemes or rules, the model represents blindly the data it was fitted on. And it works well in mechanical or chemical engineering applications if a single well delimited phenomenon, or process is represented. It can be efficient for prediction if the user does not cross (even by the smallest amount) the value domain of parameters considered and does not try to represent a situation where an additional parameter is needed (even just a single parameter more). The LLMs have basically the same limitations. But the ambition with LLMs is quite extravagant as they will pretend to describe the entire world. If one wants to describe the entire world with a “black box model”, one would need an infinite number of parameters and an infinite number of coefficients. That is not possible. That’s why obviously LLMs are not very close to AGI. In order to get AGI, or just a reliable general representation of the world complexity, we will need a hybrid approach, a combination of empirical correlations, knowledge based equations and internal strong inferring rules.
Very well said.
Yours and Gary's well-reasoned and factual opposition against hype and nonsense is failing to shake existing convictions. Exactly the same happened in the 1960s, with Life stating in a famous article in 1970 that 'electronic persons' had arrived and Minsky in that article quoted saying that AGI would be there 'within 3 to 8 years'. Minsky then. Hinton c.s. now. History doesn't repeat, but it definitely rhymes. And the voices of realism are lost in a chorus of unfounded convictions. In the discussion on artificial intelligence, the role of human intelligence and how convictions form and are resistant to argument is the real interesting subject.
You two gentlemen are way too polite. Agüera y Arcas and Norvig are not just wrong. They are miserably wrong. There is nothing in generative AI or LLMs that is even remotely useful to solving AGI. To crack AGI, the mainstream paradigm must not merely shift. It must be discarded entirely. A completely new model must arise to replace the current one. Unfortunately, I don't see this revolution coming from the deep learning generation.
PS. In my opinion, generalization and precise timing are key components of AGI.
Seems to me this was written while on auto-pilot, the temperature was a bit lower than Andreesen when he did his techno-optimist manifest, but still pretty much autopilot. You do that when you've given up on thinking for some reason or another. Maybe it's too hard, or too painful. These people just want to party down in the Good Place and leave all the worries to, well, whomever.
A more than welcome rebuttal. What I don’t get it why Arcad and Norvig would go as far as making the argument in the first place, when there is such clear evidence of the contrary...
Yes, ChatGPT is stochastic parrot.
AGI my ass!
I asked it to create a math equation homework for my daughter and solve it. Look at the output below. It has NO UNDERSTANDING of math.
ChatGPT:
5(4x - 2y + 3) + 2(3x - 5y - 6) = 20x - 10y + 15 + 6x - 10y + 12 = 26x - 20y + 27
Me:
Your math is wrong for question 10
ChatGPT:
Apologies for the mistake. Let's correct problem 10:
5(4x - 2y + 3) + 2(3x - 5y - 6) = 20x - 10y + 15 + 6x - 10y - 12 = 26x - 20y + 3
Thank you for pointing that out. If you have any more questions or need further assistance, please let me know!
Here's what he wrote:
Actually, some years later Marvin explained to me why he wanted me to
lead the project.
He said that if he put a grad student in charge the grad student would
know enough to realize that it was impossible to do what he wanted and
would just ignore him. However he wanted to know what was hard about
the problem so he put a sophomore in charge so I would just rage ahead
and tell him what difficulties I found. He said that I reported
exactly what he needed to know--that the "lines" we see in the edges
of objects in the scene are not really clearly there; the noise is
very large and much of the edges was illusory.
GJS
LLMs are in the strongly rising part of the hype cycle. It will pass, either as generative AI hits clear limits or because a new hype cycle resulting from a new approach takes over.
The terminology is kinda confusing. LLMs are a (somewhat-)general technique in AI, in that they're able to do many kinds of tasks without being specifically trained on that task, e.g. few-shot learning. But that's not what capital-G General AI means.
Anecdotally, I've heard of fine-tuning an LLM on training materials a human might use (e.g. a spec or guide) and the LLM getting better at the task. That's maybe the first time I've seen something in "AI" that we might call "learning" if done by humans. Most older supervised learning tasks (even using deep learning) would seem bizarrely rote to humans. Like a human who needed to see thousands of pictures of a cat to be able to identify a cat would be considered to have some kind of severe learning disability. But it's a big leap from "maybe not completely unintelligent" to GAI.
The success of computational, ungrounded large language models based on the distributional hypothesis tells us a great deal about about how natural languages work but probably rather little about AGI other than to confirm that, contrary to Turing's imitation game, linguistic competency (however defined) is not a good measure of general intelligence.
The observation that linguistic competency can be programmed into a box, just as numeric competency can be programmed into a calculator, challenges our view of ourselves, our understanding of what makes humans intelligent, but this isn't relevant to AGI other than perhaps gently to question what we mean by "general intelligence", as discussed in the article here.
Humans use both numeric calculations and natural language, among other techniques, to reason about the world. Machines can be programmed to do the same but that doesn't make a system like GPT-4 any more intelligent than an electronic calculator, although both can be extremely useful.
Gary, I wonder how many times we can repeat these basic weaknesses to persons unwilling to acknowledge them. Part of me thinks we are encountering something akin to a culture war within AI circles. Why not? America breeds binaries that repel all reason or logic. There is a ideological impetus undergirding the constant redefinition of AGI on the part of the corporate tech giants. What can we do? Keep our heads. Realize larger forces at work behind the debate. And focus on questions of implementation.
Gary, I love your take on this. We as a community should build a solid AGI benchmark. Or even a subset of various general intelligences benchmark (physical intelligence, verbal intelligence, game solving, mathematics e.t.c).
Right now AGI is tossed around to get that sweet sweet VC handouts. But if there is a common agreed upon benchmark, then it's easy to say we're 10% hitting AGI benchmark but failing this domains.
Unless there is objective measure, most people will talk past each other.
A good summary of the state of the art. It’s amazing how much is changed and how little has changed. Like the philosopher Dreyfus said back in 1972 and revisited in the new edition of his book (1992) “What Computers *Still* Can’t Do". 😆
This not all Just getting pretty stupid; it’s been stupid since Turing laughed at the idea. And suppose we use plasma? And get it all together? This is the most patently recognized hoax in human history. Go and write program, code it good. Or take an axe and split some wood. This is all a tautology. It is a farce. I cannot believe it. What a lament for Feyerabend: he would have been sickened. Which is not the point though. We we are wit a hoax that a hoax because it is a tautology. Bayesian LLM or any Bayesian machine needs a Constantly infinite and dynamically changing with layer of knowledge about the depth of an atom,
and the depth of the depth, And on and on as quantum mechanics advances. And, that advance is the same as the advance as taking an axe to split wood.
Take the money away from these squonks at google and IBM and distribute it to the poor and to the war-stricken and just return to sanity. “AIG”: words only, and an acronym because people like these things before they become cliches. Which AIG will. Assholes In Google? “AIG”? Fools. a hoax. Infinitely worse than Cargo Cult Science. A lament for Feynman please.