That depends on how you look. The fundamental limit is that no amount of induction (statistics) gets you deduction (symbolic logic), even if you can approximate close. The other way around is true too as we found out 40-20 years ago.
To add, AI vendors are not in principle opposed to deductive reasoning or symbolic methods. The problem is that, in practice, those are just a different kind of approximation to the messy reality. They do not solve the problems.
AGI is an immense problem, and what is in our heads is not easy to represent properly. The industry is onto something by focusing on incremental work and leveraging data as much as possible. That will inspire new directions as need be.
Symbolic logic may be seen as part of our brain's efficiency apparatus, that is, it for instance prevents infinite scaling issues (e.g. the outlier problem, but also others).
The symbolic logic people had it the wrong way around. Stuff like emotions doesn't emerge from large amounts of discrete facts and rules (the problem being that 'large' here in effect is 'infinite'), but discrete facts and rules are created out of the non-discrete, messy, chaotic, stuff below. You can create Q out of R, but not the other way around.
In that sense it may be evolutionary related to our 'conviction' efficiency apparatus, such as convictions that 'incremental work' 'will' inspire new directions 😀. Or convictions that these 'new directions' — when mentioned now — are the equivalent of vapourware.
AGI wolf calls have done their job. Anyone who believed CEOs were bearers of wisdom can take a moment, breathe, and guard their wallets until the next time Silicon Valley’s siren songs resurface.
From the ongoing timeline these A.I guys should be part of WWE hype promos! Just give it to us do not keep telling us. I will not be waiting. The human is superior for me till further notice.
There seems to be a fundamental gap in understanding how *meaning* is produced. AI systems are incapable of producing meaning, and therefore cannot achieve 'AGI' in any capacity no matter how much training data they are subjected to.
Grounded meaning requires no symbols... that's where the disconnect lies, between reality (natural intelligence) and ALL of AI-to-date including the latest dot-product-based "generative" ['computational', really] fakes.
I feel like the whole "AGI" debate is moving into absurdity. So, tech bros get to decide for everyone else with brute force? AGI is not real and has no meaning...hell it can't even be defined in contextualized linguistics/language...I think we need Noam Chomsky right now to bring some intellectual clarity and stop this science fiction nonsense.
You don't need a linguist. You need a philosopher. I recommend Robert Brandom, who wrote regarding rationality: "Rational beings are ones that ought to have reasons for what they do, and ought to act as they have reason to. They are subjects of rational obligations, prohibitions, and permissions." Given this definition it is obvious what the problem is: LLMs lack this normative dimension. They are not capable of binding themselves by what they believe or say in the way that any rational being would.
Human intelligence requires emotionality. These low eq software engineers will never get it on their own. They were over rewarded for being stoic and now confuse reward with virtue
I looks really cool, if hacky in the way that all deep learning based attempts at performing deductive logic are hacky. Notice that there is an element of deduction involved: AI-generated proposed solution steps are turned into Python code and then run to see if they actually work.
Ultimately, this is still a probabilistic "does this solution resemble solutions from the training?" approach, but done in a principled manner so as to avoid the usual problems in using language models to answer logic questions.
I'd say this is a good example of a well-established phenomenon: the more narrowly an AI is tailored to solving a specific kind of problem, the better it will perform.
GPT-5 should arrive right around the time those "few simple words" refuting Hubert Dreyfus McCarthy promised us back in the late 1960s are finally written and published.
That is a good question. I think OpenAI has enough of a clout to sell services and make a buck, eventually. What will happen to Grok, Mistral, and Zuck's vanity project is less clear.
I meant for all those capital expenditures not by the data center buildouts. I meant those corporations who are buying into all this and choosing vendors due to AI investing being requirement by c suite to not fall behind. where is their ROI?
I don't think managers are that clueless to invest in AI just because it is the fad of the day. In some industries, chatbots are useful and augment workers. If vendors like OpenAI offer tools that have value, and some customers buy them and then renew their deals, things will work out.
As usual, people who buy in blindly will not do well, and those who fail to take advantage of new functionality will not do well either. Some ongoing evaluations of the state of things is likely what they should do.
I just wrapped up a small review of research on AI agents and how they fare in high-level roles and the challenges are significant. Even if we get LLMs that are much better task planners there is something about the social level and the ability of LLMs to operate in that space with information asymmetry that seems quite a reach right now - so yeah I would be surprised if AGI showed up this year https://www.agentsdecoded.com/p/research-roundup-can-ai-agents-actually
What’s with the version numbering anyway? Isn’t it mere marketing to call it GPT 5 instead of GPT 4.7, for example? Sorry if this is a dumb question, I’m not a software developer.
Changes in the transformer architecture, I guess. Turbo was 'long context'. .5 may have been dimensional increases, like token dictionary size, embedding size, parameter volume)
Hi Gary - more and more I find articles written with AI tools - such as this one: https://www.sencha.com/blog/how-ui-components-help-developers-create-scalable-and-user-friendly-web-apps/ . These articles are dry and devoid of the meaning that other articles written with a clear intent by humans have. I was thinking of inventing a term for this type of articles, or maybe it already exists. Do you know? They are like zombies.
AGI will be here when Sam Altman decides it's time to call OpenAI's latest model "AGI". The believers will insist it meets the qualifications, the non-believers will mock it, and the term itself will finally be permitted to show the world its meaninglessness.
AI researchers would probably distinguish between intellectual AGI and human-equivalence performance. Most of the items on your April list of AGI tasks involve physical activities, e.g., "walk into random kitchens and prepare a cup of coffee or make breakfast." The three final ones, Pulitzer-caliber, Oscar-caliber, and Nobel-caliber work are the primary purely-intellectual tasks.
I agree that we don't have AI systems that are close to the purely intellectual challenges you list. But I think it's an important distinction to keep in mind.
Gary, the consummate beat reporter, has put his thumb on the matter. The Eureka! moments of random walk scaling are done. But, as Ilya Sutskever says, now is the time for 'scaling the right thing' - a more deliberative walk. Deliberation has long been fraught, but lessons have been learned in contracting to ease the way forward.
Nor will we see it in 2026, 2027, ...
(It's not an incremental issue, it is a fundamental issue. 'Wide' AI will not become'General' AI)
Intelligence is not one thing, as in you have it or you don't. It is a giant collection of skills and their seamless integration.
Machinery for high-level approximate synthesis is absolutely necessary, which is what the vendors have now.
The logic will get better in areas where there's profit. Any failures will inspire custom solutions. No fundamental limits in sight.
That depends on how you look. The fundamental limit is that no amount of induction (statistics) gets you deduction (symbolic logic), even if you can approximate close. The other way around is true too as we found out 40-20 years ago.
To add, AI vendors are not in principle opposed to deductive reasoning or symbolic methods. The problem is that, in practice, those are just a different kind of approximation to the messy reality. They do not solve the problems.
AGI is an immense problem, and what is in our heads is not easy to represent properly. The industry is onto something by focusing on incremental work and leveraging data as much as possible. That will inspire new directions as need be.
Symbolic logic may be seen as part of our brain's efficiency apparatus, that is, it for instance prevents infinite scaling issues (e.g. the outlier problem, but also others).
The symbolic logic people had it the wrong way around. Stuff like emotions doesn't emerge from large amounts of discrete facts and rules (the problem being that 'large' here in effect is 'infinite'), but discrete facts and rules are created out of the non-discrete, messy, chaotic, stuff below. You can create Q out of R, but not the other way around.
In that sense it may be evolutionary related to our 'conviction' efficiency apparatus, such as convictions that 'incremental work' 'will' inspire new directions 😀. Or convictions that these 'new directions' — when mentioned now — are the equivalent of vapourware.
Much work people do involves diligently going through steps, and checking what you get as you go. The feedback informs the next steps.
Symbolic and principled methods suffer from the same problems as LLM unless you are able to validate and model precisely what you are dealing with.
AGI wolf calls have done their job. Anyone who believed CEOs were bearers of wisdom can take a moment, breathe, and guard their wallets until the next time Silicon Valley’s siren songs resurface.
Little Red Writing Hood:
“Grandma, what big AI’s you have.”
Silicon Valley Wolf: “ The better to fleece you with my dear”
From the ongoing timeline these A.I guys should be part of WWE hype promos! Just give it to us do not keep telling us. I will not be waiting. The human is superior for me till further notice.
There seems to be a fundamental gap in understanding how *meaning* is produced. AI systems are incapable of producing meaning, and therefore cannot achieve 'AGI' in any capacity no matter how much training data they are subjected to.
Exactly.
Grounded meaning requires no symbols... that's where the disconnect lies, between reality (natural intelligence) and ALL of AI-to-date including the latest dot-product-based "generative" ['computational', really] fakes.
Grounding requires no symbols, indeed. But you have to model things somehow.
If you are arguing that none of the computational models developed for a century have no bearing on intelligence, that would require a solid argument.
That's why o3 searches around, and AI agents will invoke tools that know about meaning. One has to ground the generation, somehow.
I feel like the whole "AGI" debate is moving into absurdity. So, tech bros get to decide for everyone else with brute force? AGI is not real and has no meaning...hell it can't even be defined in contextualized linguistics/language...I think we need Noam Chomsky right now to bring some intellectual clarity and stop this science fiction nonsense.
You don't need a linguist. You need a philosopher. I recommend Robert Brandom, who wrote regarding rationality: "Rational beings are ones that ought to have reasons for what they do, and ought to act as they have reason to. They are subjects of rational obligations, prohibitions, and permissions." Given this definition it is obvious what the problem is: LLMs lack this normative dimension. They are not capable of binding themselves by what they believe or say in the way that any rational being would.
Human intelligence requires emotionality. These low eq software engineers will never get it on their own. They were over rewarded for being stoic and now confuse reward with virtue
So what do you think about things like this? https://huggingface.co/papers/2501.04519
Looks overwhelming and a little scary. Do you think it's real or they just overtrained on that exact benchmark?
I looks really cool, if hacky in the way that all deep learning based attempts at performing deductive logic are hacky. Notice that there is an element of deduction involved: AI-generated proposed solution steps are turned into Python code and then run to see if they actually work.
Ultimately, this is still a probabilistic "does this solution resemble solutions from the training?" approach, but done in a principled manner so as to avoid the usual problems in using language models to answer logic questions.
I'd say this is a good example of a well-established phenomenon: the more narrowly an AI is tailored to solving a specific kind of problem, the better it will perform.
GPT-5 should arrive right around the time those "few simple words" refuting Hubert Dreyfus McCarthy promised us back in the late 1960s are finally written and published.
And what is the ROI for all this investment?
That is a good question. I think OpenAI has enough of a clout to sell services and make a buck, eventually. What will happen to Grok, Mistral, and Zuck's vanity project is less clear.
I meant for all those capital expenditures not by the data center buildouts. I meant those corporations who are buying into all this and choosing vendors due to AI investing being requirement by c suite to not fall behind. where is their ROI?
I don't think managers are that clueless to invest in AI just because it is the fad of the day. In some industries, chatbots are useful and augment workers. If vendors like OpenAI offer tools that have value, and some customers buy them and then renew their deals, things will work out.
I have been a manager at Adobe and a Masa Softbank co. You are incorrect. They will and have.
As usual, people who buy in blindly will not do well, and those who fail to take advantage of new functionality will not do well either. Some ongoing evaluations of the state of things is likely what they should do.
I just wrapped up a small review of research on AI agents and how they fare in high-level roles and the challenges are significant. Even if we get LLMs that are much better task planners there is something about the social level and the ability of LLMs to operate in that space with information asymmetry that seems quite a reach right now - so yeah I would be surprised if AGI showed up this year https://www.agentsdecoded.com/p/research-roundup-can-ai-agents-actually
What’s with the version numbering anyway? Isn’t it mere marketing to call it GPT 5 instead of GPT 4.7, for example? Sorry if this is a dumb question, I’m not a software developer.
Changes in the transformer architecture, I guess. Turbo was 'long context'. .5 may have been dimensional increases, like token dictionary size, embedding size, parameter volume)
Hi Gary - more and more I find articles written with AI tools - such as this one: https://www.sencha.com/blog/how-ui-components-help-developers-create-scalable-and-user-friendly-web-apps/ . These articles are dry and devoid of the meaning that other articles written with a clear intent by humans have. I was thinking of inventing a term for this type of articles, or maybe it already exists. Do you know? They are like zombies.
AGI will be here when Sam Altman decides it's time to call OpenAI's latest model "AGI". The believers will insist it meets the qualifications, the non-believers will mock it, and the term itself will finally be permitted to show the world its meaninglessness.
AI researchers would probably distinguish between intellectual AGI and human-equivalence performance. Most of the items on your April list of AGI tasks involve physical activities, e.g., "walk into random kitchens and prepare a cup of coffee or make breakfast." The three final ones, Pulitzer-caliber, Oscar-caliber, and Nobel-caliber work are the primary purely-intellectual tasks.
I agree that we don't have AI systems that are close to the purely intellectual challenges you list. But I think it's an important distinction to keep in mind.
Gary, the consummate beat reporter, has put his thumb on the matter. The Eureka! moments of random walk scaling are done. But, as Ilya Sutskever says, now is the time for 'scaling the right thing' - a more deliberative walk. Deliberation has long been fraught, but lessons have been learned in contracting to ease the way forward.
"Gary Marcus wishes the media would hold those who make unrealistic promises to account. Because they *rarely do...." right?