Even if we agree that AGI is relatively imminent, the question is would that AGI merely be an engineering achievement or would it also shed some scientific light on how human (or biological) intelligence works? If it would be nothing more than engineering feat then trying to achieve AGI is also off-ramp from the goals set for themselves by the founders of Artificial Intelligence.
Well said. They don't seem to care the rush is to get to something that appears and acts intelligent enough that it take over ever more complex tasks. If people think it's a human calling from Canads Revenue Agency to discuss my tax return, that will be the gold stsndard even if they don't reslky understand how it works or even if it is 100% safe and reliable. Never mind sentience, understanding, compassion or love.
You'll also have more time to read poetry written by machines. Cos who doesn't want some of that? :)
Listening to how some AI people talk about art makes me sad. They don't see it as a way for human beings to connect with each other. Art is an expression of human creativity; this element is fundamental to my experience of taking it in. It's not just about technique and skill, it's about the fact that another person chose to make it in the way they did. I can't imagine reading poetry or listening to music or looking at paintings and not feeling this.
Hi Gary, I've recently been following your writing ever since I noticed that progress past GPT4 has been minimal for the most part. I was impressed with the audio2audio capabilities of 4o, but was disappointed that it didn't come with a corresponding increase in reliability or reasoning.
I think it's fair to say that there has been essentially no base improvements in either reliability or reasoning since GPT 4, which bodes very badly for the scaling laws crowd which has begun admitting we can't scale to anything like AGI. Are you planning on doing a more thorough response to Leopold's situational awareness articles? You did already write about the most important/absurd part of his first article, the graph purporting compute to human intelligence (and also claiming 4 is as smart as a smart high schooler). Keep up the great work.
"[...] which bodes very badly for the scaling laws crowd [...]"
You see, we can safely assume that scaling is dead because of the last 1.5 years. The argument is as follows: Irrespective of how much ressources OpenAI used for ChatGPT (we know it wasn't *that* much to begin with), there have been models trained with as much as six orders of magnitude larger complexity[1]. Not because of Moore's law, but because the increased interest allowed to scale the invest (not so much the tech). And the improvement we saw was absolutely miniscule. No, scaling isn't dead, it was never alive and all there is to it is the fact that LLMs passed the threshold of gramatically correct sentences with BERT and GPT-2. But you can achieve *that* with consumer grade hardware.
[1] Consider scaling up model params 100x, training data 100x, epochs / RHF 100x...
I looked at open AIs scaling law paper and graphed their scaling law for loss versus number of parameters. It plateaus relatively quickly. So their scaling laws predict no great increase in performance.
https://ea.rna.nl/2024/02/13/will-sam-altmans-7-trillion-ai-plan-rescue-ai/ shows the plateauing as well. For benchmarks that have to do with real understanding such as Winogrande the numbers are horrendous. There is a small caveat, but not one that changes the conclusion (still have to write that one up)
Guessing “AGI” experts intend to strip individuality (among specie populations; why stop at just one) out of “their version” of this upcoming, multi-defined/labeled system.
When everything else fails: move the goalpost, the ball, the kicker — anything so you still seem competent and knowledgeable.
"anything you can define will be achieved because AI keeps getting better and better"
Which AI? Is he talking about LLMs or the broad general notion of AI? If LLMs, then is he suggesting ChatGPT will have a cure for cancer? That's pretty well defined? How about telepathy? I think there's a good definition of that too.
It’s worth to read the resent books published by Federico Faggin, the ‘father‘ of modern microprocessors (Fairchild/Intel/Zilog). I guess that Federico would agree when I say that all these AGI predictors are suffering from a very basic ontological misunderstanding of what ‘consciousness’ is all about. Highly recommended to read, as Federico comes from hard-core physics and builds bridges to the understanding of consciousness.
It's funny Kurzweil uses poetry as an example, because artistic originality is one of the things generative AI is incapable of. You can train an AI to make paintings, but it will never invent a new style of painting. You can train an AI to generate music, but it will never invent a new genre of music.
This is my top choice for those "goalposts" AI boosters love to mention. When they ask what it would take for me to say an AI had human-level intelligence, my answer is "when AI can create completely original art". Good luck brute-forcing that one with bigger models and datasets and compute :)
Writing top-notch poetry as a standard for AGI? This leaves a galactic size wiggle room for subjective interpretation. Corporate PR will have a lot of fun with it before an earnings conference call.
It seems actually quite fitting go describe LeCun's position as having left the 'scale solves everything and is doable' sinking ship, versus those that knew beforehand that this ship would not fly (it might float, but fly...?). It shows that he is at least intelligent enough, and in effect critiquing your own previous position is in fact often most impressive (see later Wittgenstein versus earlier Wittgenstein for what can be considered one of the most impressive examples). So, kudos for Yann, though actually publicly admitting you yourself had the other position in the first place is part of what makes it truly impressive, so he's not entirely there yet...
Kurzweil is a true cultural (see Dave Karpf who looked into the Wired culture of the 1990s) product of the 'beginning of the S-curve' (which deceptively looks like an exponential, so a short sighted belief in Moore's Law overrides any critical thought) and one of the many simpletons that are influential (Hinton, Musk, Sutskever, ...). He chooses poetry as the yardstick. That is a safe bet forGenAI that it can do that. As well as other creative stuff where reliability and truthfulness is not necessarily a must. But 'good' it will not become, we will have 'mediocre', 'middle of the road', 'cheap' (and lots of it). I would say that a truly autonomous car would be a good yardstick (still think it is), or going into an unknown kitchen and preparing food (no bleach allowed).
You know, I wonder, can we ask Google to reactively classify all the YouTube videos of 2023 ( year ago, the fever of GPT-5 was particularly running high) that talk about GPT-5,6,7..., Q*,AGI as disinformation?
he pretty much was until about 2015 and you can see it in his earlier talks and 2015 Nature paper. and that thread was still active when he critiqued my critique of LLMs in 2019 and 2022.
LeCun's 2015 paper (https://www.nature.com/articles/nature14539) was about end-to-end-neural nets for "speech recognition, visual object recognition, object detection and many other domains such as drug discovery".
For those, "scale" makes sense.
What changed, since then, is that people want to solve AGI. For those scale is not enough, and LeCun never pretended it was.
LeCun's critique of your critique was about the fact that he believes that neural nets have to model things at a deeper level than language, and you believe neural nets are not even the answer.
In that thread (somewhere in the comments when someone asked that exact question about LeCun and scaling) I shared https://youtu.be/vdWPQ6iAkT4 The subject of the — then still friendly — debate is about requiring more than the fitting that NNs can do and I recall there he says at one point 'very little' or something to that effect (my impression is that at that point nobody thought of explicitly excluding AGI from the debate, it was generally implicit). The only way open that is then left to get better is *more* fitting (i.e. scaling). I don't know if Yann LeCun was ever as blunt as the somewhat simplistic Sutskevers etc. on this (probably not, he seems far too intelligent for that — not that I agree with everything he proposes now).
Disagreeing about "More Innate Machinery" does not mean "scale alone".
LeCun is saying that "you want to minimize the amount of prior knowledge". That makes sense, as it is very hard to collect the prior knowledge, as put in by people, it will never be enough, and it will result in a fragile system.
The ideal system would have very general representations, and will learn how to fill in the details on its own.
The debate is far from over on these things. In either case, LeCun is not claiming that mindless adding data will solve everything. He favors creating models, but the models would learn on their own.
The internet was about to take our jobs, every five years.
The election was about to end democracy, every four years.
Teslas were supposed to safely drive themselves, every other year.
Silicon Valley, Wall Street and Madison Avenue must have a new Chia pet, every couple of years. Otherwise people might learn to, you know, just get on with their lives.
Honestly, because of the debt driven nature of the world. Debt, which has now become reduced to the drive of our oil metabolism. Paradoxically, we may very well be approaching the limit of what an oil metabolism can meaningful do, as evidenced by the Ouroboros nature of the hype system.
My hope is that this is a bottleneck we break through on. But I don't assume to know the future.
Oh, Ray, why are you choosing to die on the poetry hill?
Poetry works best not on the accumulation of explicit connections and context between words. It works best on their absence.
It works best when normal words fail to convey explicit linguistic meaning and experience in our lives. Where we instead infer what emerges when words are placed in novel constructs where hidden meanings take on implicit form.
Hard to be sure but my take on Murati's "they’re not that far ahead” is that she's hinting that the release of wonderful advanced AI (hinting AGI without saying "AGI") is just around the corner. She's not saying that OpenAI's unreleased products are not much of a technical advance. She's just doing more marketing hocus-pocus, enabling the AI fanboys to talk about coming AGI without explicitly making that promise.
Murati's comment is just pure spin. She is trying to say "there's no AGI monster in our lab". She surely believes that OpenAI is still ahead, and the next products will maintain the lead.
These people are always spinning but I think she’s still allowing people to believe that AGI is some soon-to-be-released OpenAI product. This is something I hear repeated often by AI fanboys on Reddit. Based on this short clip, I don’t see how you can spin that into "there's no AGI monster in our lab". Is there a longer clip of Murati’s speech?
She says their models in the lab are not that more capable than there already is. I think that suggests that the progress is more incremental than what some people think.
Watch the first two minutes of what you shared with me. Her exact words: "models in the lab are not that more capable than what people have access to".
She's talking about the models that are coming out soon. It doesn't at all counter the rumor that they have AGI coming. Her response to the interviewer's single mention of "AGI" was to simply say that it will be "incredibly transformative technology". She had a chance to say something about AGI but said nothing.
Here's the interviewer question and Murati's answer. The interviewer is the only one who mentions AGI. Murati never mentions it but just talks about soon to be released wonderful technology. She had her chance to address the "AGI monster" and passed. IMHO, they are purposefully ignoring the elephant in the room.
Appreciate you, Gary. An honest voice among zealots who have so much invested financially, mentally, and emotionally that they can't think straight.
Even if we agree that AGI is relatively imminent, the question is would that AGI merely be an engineering achievement or would it also shed some scientific light on how human (or biological) intelligence works? If it would be nothing more than engineering feat then trying to achieve AGI is also off-ramp from the goals set for themselves by the founders of Artificial Intelligence.
Well said. They don't seem to care the rush is to get to something that appears and acts intelligent enough that it take over ever more complex tasks. If people think it's a human calling from Canads Revenue Agency to discuss my tax return, that will be the gold stsndard even if they don't reslky understand how it works or even if it is 100% safe and reliable. Never mind sentience, understanding, compassion or love.
Just what we need: machines to write poetry for us so we will have more time to sweep floors and do laundry.
You'll also have more time to read poetry written by machines. Cos who doesn't want some of that? :)
Listening to how some AI people talk about art makes me sad. They don't see it as a way for human beings to connect with each other. Art is an expression of human creativity; this element is fundamental to my experience of taking it in. It's not just about technique and skill, it's about the fact that another person chose to make it in the way they did. I can't imagine reading poetry or listening to music or looking at paintings and not feeling this.
ROTFL
Hi Gary, I've recently been following your writing ever since I noticed that progress past GPT4 has been minimal for the most part. I was impressed with the audio2audio capabilities of 4o, but was disappointed that it didn't come with a corresponding increase in reliability or reasoning.
I think it's fair to say that there has been essentially no base improvements in either reliability or reasoning since GPT 4, which bodes very badly for the scaling laws crowd which has begun admitting we can't scale to anything like AGI. Are you planning on doing a more thorough response to Leopold's situational awareness articles? You did already write about the most important/absurd part of his first article, the graph purporting compute to human intelligence (and also claiming 4 is as smart as a smart high schooler). Keep up the great work.
"[...] which bodes very badly for the scaling laws crowd [...]"
You see, we can safely assume that scaling is dead because of the last 1.5 years. The argument is as follows: Irrespective of how much ressources OpenAI used for ChatGPT (we know it wasn't *that* much to begin with), there have been models trained with as much as six orders of magnitude larger complexity[1]. Not because of Moore's law, but because the increased interest allowed to scale the invest (not so much the tech). And the improvement we saw was absolutely miniscule. No, scaling isn't dead, it was never alive and all there is to it is the fact that LLMs passed the threshold of gramatically correct sentences with BERT and GPT-2. But you can achieve *that* with consumer grade hardware.
[1] Consider scaling up model params 100x, training data 100x, epochs / RHF 100x...
I looked at open AIs scaling law paper and graphed their scaling law for loss versus number of parameters. It plateaus relatively quickly. So their scaling laws predict no great increase in performance.
https://ea.rna.nl/2024/02/13/will-sam-altmans-7-trillion-ai-plan-rescue-ai/ shows the plateauing as well. For benchmarks that have to do with real understanding such as Winogrande the numbers are horrendous. There is a small caveat, but not one that changes the conclusion (still have to write that one up)
Gary is a good source. You also might find these helpful: https://ea.rna.nl/the-chatgpt-and-friends-collection/
Guessing “AGI” experts intend to strip individuality (among specie populations; why stop at just one) out of “their version” of this upcoming, multi-defined/labeled system.
When everything else fails: move the goalpost, the ball, the kicker — anything so you still seem competent and knowledgeable.
Wait, can we digest Kurzweil's quote a moment?
"anything you can define will be achieved because AI keeps getting better and better"
Which AI? Is he talking about LLMs or the broad general notion of AI? If LLMs, then is he suggesting ChatGPT will have a cure for cancer? That's pretty well defined? How about telepathy? I think there's a good definition of that too.
It’s worth to read the resent books published by Federico Faggin, the ‘father‘ of modern microprocessors (Fairchild/Intel/Zilog). I guess that Federico would agree when I say that all these AGI predictors are suffering from a very basic ontological misunderstanding of what ‘consciousness’ is all about. Highly recommended to read, as Federico comes from hard-core physics and builds bridges to the understanding of consciousness.
I just ordered it, as a gift for my daughter. Maybe she'll lend it to me when she's done.
Or even going further back to von Neumann's The Computer and the Brain.
Thanks I will check him out. He collaborated with the mighty Roger Penrose, so clearly he has some major smarts!
It's funny Kurzweil uses poetry as an example, because artistic originality is one of the things generative AI is incapable of. You can train an AI to make paintings, but it will never invent a new style of painting. You can train an AI to generate music, but it will never invent a new genre of music.
This is my top choice for those "goalposts" AI boosters love to mention. When they ask what it would take for me to say an AI had human-level intelligence, my answer is "when AI can create completely original art". Good luck brute-forcing that one with bigger models and datasets and compute :)
Writing top-notch poetry as a standard for AGI? This leaves a galactic size wiggle room for subjective interpretation. Corporate PR will have a lot of fun with it before an earnings conference call.
yeeeep!
It seems actually quite fitting go describe LeCun's position as having left the 'scale solves everything and is doable' sinking ship, versus those that knew beforehand that this ship would not fly (it might float, but fly...?). It shows that he is at least intelligent enough, and in effect critiquing your own previous position is in fact often most impressive (see later Wittgenstein versus earlier Wittgenstein for what can be considered one of the most impressive examples). So, kudos for Yann, though actually publicly admitting you yourself had the other position in the first place is part of what makes it truly impressive, so he's not entirely there yet...
Kurzweil is a true cultural (see Dave Karpf who looked into the Wired culture of the 1990s) product of the 'beginning of the S-curve' (which deceptively looks like an exponential, so a short sighted belief in Moore's Law overrides any critical thought) and one of the many simpletons that are influential (Hinton, Musk, Sutskever, ...). He chooses poetry as the yardstick. That is a safe bet forGenAI that it can do that. As well as other creative stuff where reliability and truthfulness is not necessarily a must. But 'good' it will not become, we will have 'mediocre', 'middle of the road', 'cheap' (and lots of it). I would say that a truly autonomous car would be a good yardstick (still think it is), or going into an unknown kitchen and preparing food (no bleach allowed).
You know, I wonder, can we ask Google to reactively classify all the YouTube videos of 2023 ( year ago, the fever of GPT-5 was particularly running high) that talk about GPT-5,6,7..., Q*,AGI as disinformation?
LeCun was never in the camp of "scale solves everything".
Kurzweil's predictions did not fare well. In 1999 he predicted self-driving cars in 2010.
he pretty much was until about 2015 and you can see it in his earlier talks and 2015 Nature paper. and that thread was still active when he critiqued my critique of LLMs in 2019 and 2022.
LeCun's 2015 paper (https://www.nature.com/articles/nature14539) was about end-to-end-neural nets for "speech recognition, visual object recognition, object detection and many other domains such as drug discovery".
For those, "scale" makes sense.
What changed, since then, is that people want to solve AGI. For those scale is not enough, and LeCun never pretended it was.
LeCun's critique of your critique was about the fact that he believes that neural nets have to model things at a deeper level than language, and you believe neural nets are not even the answer.
Effectively he more or less was. See this thread: https://www.linkedin.com/posts/gerbenwierda_openai-cto-responds-to-elon-musk-calling-activity-7207725724114272256-lBvU
Your link does not show that LeCun was in the camp of "scale solves everything".
In that thread (somewhere in the comments when someone asked that exact question about LeCun and scaling) I shared https://youtu.be/vdWPQ6iAkT4 The subject of the — then still friendly — debate is about requiring more than the fitting that NNs can do and I recall there he says at one point 'very little' or something to that effect (my impression is that at that point nobody thought of explicitly excluding AGI from the debate, it was generally implicit). The only way open that is then left to get better is *more* fitting (i.e. scaling). I don't know if Yann LeCun was ever as blunt as the somewhat simplistic Sutskevers etc. on this (probably not, he seems far too intelligent for that — not that I agree with everything he proposes now).
Disagreeing about "More Innate Machinery" does not mean "scale alone".
LeCun is saying that "you want to minimize the amount of prior knowledge". That makes sense, as it is very hard to collect the prior knowledge, as put in by people, it will never be enough, and it will result in a fragile system.
The ideal system would have very general representations, and will learn how to fill in the details on its own.
The debate is far from over on these things. In either case, LeCun is not claiming that mindless adding data will solve everything. He favors creating models, but the models would learn on their own.
That is a good correction.
There is a question about 'how' that lies behind this. Where do 'general representations' come from and what does 'on its own' mean?
Poetry as an AI test? Where have I heard that before... oh yeah, this in this classic
https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf
The relevant quote:
This argument is very, well expressed in Professor Jefferson's Lister Oration for 1949,
from which I quote. "Not until a machine can write a sonnet or compose a concerto
because of thoughts and emotions felt, and not by the chance fall of symbols, could we
agree that machine equals brain-that is, not only write it but know that it had written it.
No mechanism could feel (and not merely artificially signal, an easy contrivance)
pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made
miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get
what it wants."
This argument appears to be a denial of the validity of our test. According to the most
extreme form of this view the only way by which one could be sure that machine thinks
is to be the machine and to feel oneself thinking. One could then describe these feelings
to the world, but of course no one would be justified in taking any notice. Likewise
according to this view the only way to know that a man thinks is to be that particular
man. It is in fact the solipsist point of view. It may be the most logical view to hold but it
makes communication of ideas difficult. A is liable to believe "A thinks but B does not"
whilst B believes "B thinks but A does not." instead of arguing continually over this point
it is usual to have the polite convention that everyone thinks.
I am sure that Professor Jefferson does not wish to adopt the extreme and solipsist point
of view. Probably he would be quite willing to accept the imitation game as a test. The
game (with the player B omitted) is frequently used in practice under the name of viva
voce to discover whether some one really understands something or has "learnt it parrot
fashion." Let us listen in to a part of such a viva voce:
Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a
summer's day," would not "a spring day" do as well or better?
Witness: It wouldn't scan.
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day.
Interrogator: Would you say Mr. Pickwick reminded you of Christmas?
Witness: In a way.
Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would
mind the comparison.
Witness: I don't think you're serious. By a winter's day one means a typical winter's day,
rather than a special one like Christmas.
And so on, What would Professor Jefferson say if the sonnet-writing machine was able to
answer like this in the viva voce? I do not know whether he would regard the machine as
"merely artificially signalling" these answers, but if the answers were as satisfactory and
sustained as in the above passage I do not think he would describe it as "an easy
contrivance." This phrase is, I think, intended to cover such devices as the inclusion in
the machine of a record of someone reading a sonnet, with appropriate switching to turn
it on from time to time.
Just a little typo to fix here: "Others are bailing now to." Never got on board the LLM craze.
We live in a culture of hype.
The planet has been about to end, every 12 years.
The internet was about to take our jobs, every five years.
The election was about to end democracy, every four years.
Teslas were supposed to safely drive themselves, every other year.
Silicon Valley, Wall Street and Madison Avenue must have a new Chia pet, every couple of years. Otherwise people might learn to, you know, just get on with their lives.
Honestly, because of the debt driven nature of the world. Debt, which has now become reduced to the drive of our oil metabolism. Paradoxically, we may very well be approaching the limit of what an oil metabolism can meaningful do, as evidenced by the Ouroboros nature of the hype system.
My hope is that this is a bottleneck we break through on. But I don't assume to know the future.
“Anything you can define will happen because AI will get better and better”. This statement says it all. It’s pure sophistry.
Oh, Ray, why are you choosing to die on the poetry hill?
Poetry works best not on the accumulation of explicit connections and context between words. It works best on their absence.
It works best when normal words fail to convey explicit linguistic meaning and experience in our lives. Where we instead infer what emerges when words are placed in novel constructs where hidden meanings take on implicit form.
AI by nature violently despises novelty.
Hard to be sure but my take on Murati's "they’re not that far ahead” is that she's hinting that the release of wonderful advanced AI (hinting AGI without saying "AGI") is just around the corner. She's not saying that OpenAI's unreleased products are not much of a technical advance. She's just doing more marketing hocus-pocus, enabling the AI fanboys to talk about coming AGI without explicitly making that promise.
Murati's comment is just pure spin. She is trying to say "there's no AGI monster in our lab". She surely believes that OpenAI is still ahead, and the next products will maintain the lead.
Love that the best defense of OpenAI is that they are lying lol
These people are always spinning but I think she’s still allowing people to believe that AGI is some soon-to-be-released OpenAI product. This is something I hear repeated often by AI fanboys on Reddit. Based on this short clip, I don’t see how you can spin that into "there's no AGI monster in our lab". Is there a longer clip of Murati’s speech?
She says their models in the lab are not that more capable than there already is. I think that suggests that the progress is more incremental than what some people think.
Where exactly does she say that? Not in the interview I watched.
Watch the first two minutes of what you shared with me. Her exact words: "models in the lab are not that more capable than what people have access to".
She's talking about the models that are coming out soon. It doesn't at all counter the rumor that they have AGI coming. Her response to the interviewer's single mention of "AGI" was to simply say that it will be "incredibly transformative technology". She had a chance to say something about AGI but said nothing.
Here's the interviewer question and Murati's answer. The interviewer is the only one who mentions AGI. Murati never mentions it but just talks about soon to be released wonderful technology. She had her chance to address the "AGI monster" and passed. IMHO, they are purposefully ignoring the elephant in the room.
https://youtu.be/BD0Us5Bn6Lw?si=L1WxKPse4-G8o0_R&t=753