So, watching the full interview (so you all don't have to), Altman doesn't say that a new architecture is needed for AGI, but he states that we already are very close to AGI (within 2 years) and AGI itself will help to find *another* big breakthrough for (that) AGI.
- (22:07): "I think if you are a sophomore now, you will graduate to a world with AGI in it" (i.e. AGI in 2 years)
- (22:44) "Science will be automated" (in the coming years)
- (26:35) Takes moltbook seriously
- (27:30) "robots won't be that far off" (in the coming years)
- (28:52) "It will feel like we've broken the laws of traditional speed and product development and business growth rates [...] companies will start to hire AI-coworkers and do amazing things with fewer people" — echoes "New Economy" to me and paging Brian Merchant...
- (30:20) Defines AGI as a system that "could do better at making decisions than anyone we could possible have be like say president of the United States" (Um, that is an interesting benchmark these days, but let's not digress)
- (33:22) "the jobs 100 years of from now look nothing like the jobs of today, nor do that many of the jobs of today look like the jobs of a few hundred years ago.". This one is really interesting, because that argument sounds pretty convincing until you notice he is talking about a timescale of hundreds of years...
- (36:56) "We are burning cash very fast. It is not a serious situation [...] if we can spend a billion dollars this year to make $3 billion next year, there's a lot of capital in the world that wants to do that. So we will raise a lot of money, equity and debt" (IPO coming up, but that was clear already). He's promising 200% return on investors in a year.
The actual statement on a new architecture is an answer to a question "What AI subfield today feels like OpenAI in 2016, a little undervalued or underestimated but could shoot out of nowhere?" Then he answers 'from the research perspective': "I bet there is another new architecture to find that is going to be like a big gain as transformers were over LSTM. And I think you finally have models that are smart enough to help do that kind of thing". So we are already close to AGI and that will speed up the improvement of AGI even further.
Basically it is the same as always, no real change. The beginning was also interesting. Sam is a believer in this, he has been smitten by the improvements from transformers and he has translated that rapid curve up into an exponential curve, not the beginning of the S-curve which it almost certainly is.
THREE THINGS: First, thank you for the outline--very helpful, though I want to view the video also.
SECOND, about: <- (22:44) "Science will be automated" (in the coming years)> There is a "catch" here that needs unraveling. That is, does Altman mean the science of the machine/ artificial part of intelligence, e.g., the physical sciences of it? OR does he refer to the human sciences that have an enormous influence, not on the physics but on the content?
I have written here before that it seems to me that the narratives confuse the natural elements of humans (our physics, biology, etc.) with the now much more mature social and other sciences (and the arts and history, philosophy, theology) all of which move "up" from the natural/physical to the normative that relates, (basically) to what (ironically as natural) is consciously "agentic" about human beings. And so the question becomes, what exactly informs and provides the basic dynamics of the albeit-modified freedom of thought, deliberate choices, and action that human beings regularly exhibit?
Or to put it more simply, one does not develop emotionally, intellectually, socially, ethically, politically, or spiritually by merely hearing sound (rather than language), putting a bandage on a cut or, eating vegetables rather than dirt.
And THIRD< about: <Then he answers 'from the research perspective': "I bet there is another new architecture to find that is going to be like a big gain as transformers were over LSTM. And I think you finally have models that are smart enough to help do that kind of thing". >
Does he mean only ONE architectural element? or are there perhaps many that, first, develop differently in history, and that do not hail back only to the physical/natural sciences as their mode, but to massive elements that concern human normativity?
I don’t think Sam is that thorough in his thinking, it vvseems all very shallow and fuzzy to me. I don’t think people should pay too much attention to what he says.
Altman does not understand much of anything. One should understand him as an embodiment of the cult of mediocrity that dominates much of the world at the moment. Altman does not have a degree in physics or in any other kind of natural or social science; rather, if I recall correctly, he studied undergraduate computer science, dropping out after two years because he was too cool for school and wanted to play startup with his parents' money. Like the other members of this cult of mediocrity, he profits from work done by people who are, in many cases, far from mediocre, and what little knowledge he himself has is secondhand at best. When he says something, it should be understood as somewhere between a lie, a confident assertion of his ignorance, or in the best case, a distorted recollection of what someone actually knowledgeable working at the company told him.
This is nothing more than a case in point. Regardless of whether current transformer models are capable of doing everything that scientists can, most physicists believe that one of the largest obstacles to developing a complete theory of physics is that there are interactions that occur so rarely, so quickly, at such small resolutions, or at such high energies (and many of these things are related), that either a great deal of time, extremely large measurement devices, or both, would be needed to distinguish between competing theories that are equally consistent with current evidence. Perhaps chatbots could come up with some very clever idea to avoid these limitations, but perhaps not. Unless the clever idea is taking over the world and building a particle accelerator out to the orbit of Jupiter, which is something I do think Altman and his crew might be foolish enough to do by accident.
However, it's hardly "deep thinking" to realize the difference between
(a) physics/nature and their empirically established and replicate laws and protocols as guides for platform building, and the folly of
(b) applying them directly to human content where human beings and their normativity is what informs much of what is scaled, and that, though
(c) human content albeit standing on sentient experience governed by physics and nature in the time-space continuum, still raises up to different levels and gradients to define even past and future speculations about what we mean by wholly human experience and reality at different moments in human developmental patterns.
"AGI itself will help to find *another* big breakthrough for (that) AGI." ... So in order to have AGI we first need AGI to help us invent it? Cool, cool, cool.
No, he doesn't say anything about kinds of breakthroughs further down. Might be supercheap AGI for instance, might be anther improvement. The statement isn't about *reaching* AGI at all, that, he says, is going to happen in the coming two years on the current transformer trajectory.
He is just recycling I. J. Good's increasing machine capability as each more powerful iteration is built by the previous one, heading towards the singularity. Of course, he might just have read The Hitchhiker's Guide to the Galaxy books and recalled that to answer the question of "What is the answer to the ultimate question", the supercomputer Deep Thought designed the next more powerful computer...the Earth.
The tell isn't the admission of uncertainty—it's the continued spending. Altman can say 'we need new architectures' and mean it, while still building 00B data centers for the current one. Institutional momentum doesn't reverse on honest reappraisals; it just absorbs them. The real concession would be a change in what gets funded, not a change in what gets said.
Well, maybe the dawning realisation that OpenAI and Anthropic will not be able to meet their financial obligations will get through the poisonous AI-hype-fog much quicker than any other reconsideration. Being hit in the purse hurts the most...
The conversation about AGI obscures the real danger of LLMs and massive data centers. The LLMs are extraordinarily good at sifting through massive amounts of data. Essentially they are search engines on steroids. They are very useful tools in the hands of individuals using them for benign or good purposes. They are very dangerous tools in the hands of power seeking individuals.
Even with exponential growth in hardware processing, traditional computing methods and software it was really not possible to do mass surveillance of a population of 330+ million people. The LLMs (aka AIs) have changed that. You are no longer anonymous because you were one person in vast mass of data. The LLMs can find a needle in a massive haystack in seconds.
Based on some back of the envelope calculations, it appears that a typical Gemini cluster could process in real time the combined voice and text output from roughly 250,000 people. Google owns roughly enough compute infrastructure to host several hundred Gemini clusters. I don't know anything about what the other companies have but the LLMs and data centers in existence today would have little difficulty identifying and then tracking in real time the text and voice output of the 3 to 10 million people most likely to be active resisters of an authoritarian government.
I think the real danger of "AI" is not an AGI that will destroy humanity. It's surveillance and data processing (aka LLMs) technology in the hands of an authoritarian government who can use the force of government to control people who would most actively resist an authoritarian takeover like what we're seeing today.
Control of masses of people ultimately comes down to the physical force of government applied to people resisting that government who can be identified and tracked by that government. We've seen this administration centralize data, extract that data, support the building of large scale data centers for use by government to process that data, fund a national paramilitary force, and fund defacto concentration camps.
I don't think this government cares about creating an AGI - it has what it wants right now - the ability to identify and control.
It's starting to feel a lot like we are starting to live the 2011 TV series Person of Interest, which was science fiction then. Now, it's not far from the technical reality of 2026.
Jim Cook: Don't forget the leverages for getting "there," e.g., extortion, bribery, threats to one's family (apparently used during the first and second impeachments) and the old standby: high-rise windows.
Your assumption about how good the current LLM models is close but not quite there. Models do inference. Search + RAG are helpful I. Fimdong source material. However, a recent Stanford AI research (published in the Journal of Empirical Legal Studies) paper show how this fails as more data is added the the accuracy of relevant sources (documents) declined at scale. It was consistent and repeatable. Still much room to improve.
Did a quick scan but have to go back and read it more carefully.
I agree about much room to improve. I've been using the cheapest non-free version of Gemini as an augmentation to basic web searching. I've noticed multiple instances of it making up links and getting fairly basic information wrong. When corrected, it "apologizes" and usually provides correct information. However, I've found a few instances where it continues to provide the same incorrect response no matter how much I correct it. My overall take is it's a useful tool (generally saves me time over a simple search query) but you do have to verify information presented. I've also had some success with providing explicit instructions to verify every link, never give me an invalid link and provide something like "<Link not found/available>" instead.
These are just tools, not "intelligences". You outsource your intelligence to a tool at your own risk. I'm a retired attorney and would not use any citation from a LLM without verifying the citation and that the case actually supported the position I was advocating. New attorneys just using the output without verifying are putting their careers at risk, not to mention the harm to the clients they supposedly represent.
I don't think the errors are much of a concern to an authoritarian government using the tools for surveillance/control. Errors causing misidentification, etc. (See: https://www.businessinsider.com/flock-safety-alpr-cameras-misreads-2026-3) may actually be a feature rather than a bug because reports of the negative impacts are likely to make people more concerned about how resistance could harm them. Basically the goal is suppression and if a few people get hurt along the way, the people in power are OK with that.
And exactly where does one go to present/develop/refine/discuss such a new architecture?! Places like NeurIPS and ICLR seem stuck in looking for statistical approaches ONLY. Exeter Uni’s SiC 2025 is so far the only place I’ve found that seems open, but also seems poorly equipped (intellectually) for the task: https://youtu.be/eTLMu_CEa2E?is=udraKihpC4dVeOel
The politicians and tech bros who so desperately want AGI as fast as possible dont want AGI per say, they want control. The evidence is more and more clear. This is a "people problem" at 10x scale.
Ultimately, the more people wake up to the reality of this, and the more people realize the marketing hype and fear is a lie, the faster we can get out of this nightmare.
AI was here, is here, and will continue to be here.
But that doesnt mean we have to blindly accept enshitification of everything because of it.
There has been so much hype, so many lies, and so many affirmations from these characters that I feel they suffer from a short memory and a broken record syndrome. Perhaps they have become what their models represent? Indeed, new architectures are required, but we also require new people outside this realm of hallucinated fantasy. We need responsible, accountable, and truthful people. He, and many others, do not seem to belong to this category.
Slippery Sam’s little escape clause — effectively, that current AI will find that new architecture — is the last — rather contradictory — line of defence.
I wonder if this is part of setting the stage for the IPO, so the current shareholders would like to recoup as much as they sill can by fleecing the retail investors. Claiming they are near to AGi would probably be legally problematic, so this way they still sort of can.
Or maybe, just maybe, we could stop digging the hole deeper and wash our hands of this whole sorry project. It's past time for us to wake up from this mad, bad, dystopian feverdream.
I am ok with no AGI for now. They do useful things. The real question is do they do enough to justify the costs and investments? Probably that’s a “sort of”…
GPUs are also good at simulations (classic stuff, not AI).
If we can repropose the ones already built for more generic workloads and cut down the cost for 3D simulation and optimization of various objects we use and need (FEM, geometric optimization, global optimization) then the money won't be wasted.
Hm, I don't think it is that easy. Yes, GPUs are used in simulations. The problem might be the accuracy and resolution of the data types the GPUs are optimized for. So the type of data AI GPUs are optimized and built for might not be suitable, i.e. accurate enough for weather or nuclear simulations.
Yeah, that is a problem. But the GPU architecture structure can be changed in the data center to accommodate different workloads.
LLMs will not go anywhere. They are overhyped but still useful if used carefully in specific use cases where verification of the output is possible to be done automatically. Code generation is such a domain.
There is stll the problem of the extreme inefficiency from energy perspective but it can be mitigated at least partially by using dedicated hardware.
Interesting enough, but in many SciFi movies the “advanced” computing is based on crystals. Now, here's the thing, if you wish to optimize a trained ANN you can “set it in stone” by building the neural net including the weights in a “chip tower” that can be fancifully shaped as a large crystal in order to also disipate energy.
So, SciFi is Sci because it has less “Fi” and usually can be realized in a future ahead of us.
Again, changing the GPU architecture is not going to be easy or cheap, as you would have to replace the whole racks with their bespoke cooling and power provisioning. This would not be like replacing ordinary servers with a different model.
Also, actually running the GPUs needs tasks where such costs are not relevant. Nuclear simulation, of course, but *normal* tasks would have to be chosen very carefully to ensure that the costs are way below of actual value created.
It would be funny if it wasn’t so tragic
honestly, it’s both, funny and tragic.
BREAKING: Donald Trump concedes that we need major breakthroughs beyond mere bombing to get to regime change
Don't you wish!
Uhh, regime change WHERE exactly- at the White House?!
Yeah, but now the work environment is ruined, the internet is ruined, the economy is ruined and the world prospect itself looks ominous
Lets' not forget massive copyright infringement and a crisis for intellectual property, John.
Perhaps it is time for high power lawyers to start sharpening their pencils and claws. Circle above the dying beast...
There’s an AI model for that.
Nope. It is transformed.
Into something which if we survive, will make them stronger.
The LLMs will continue until morale improves.
So, watching the full interview (so you all don't have to), Altman doesn't say that a new architecture is needed for AGI, but he states that we already are very close to AGI (within 2 years) and AGI itself will help to find *another* big breakthrough for (that) AGI.
- (22:07): "I think if you are a sophomore now, you will graduate to a world with AGI in it" (i.e. AGI in 2 years)
- (22:44) "Science will be automated" (in the coming years)
- (26:35) Takes moltbook seriously
- (27:30) "robots won't be that far off" (in the coming years)
- (28:52) "It will feel like we've broken the laws of traditional speed and product development and business growth rates [...] companies will start to hire AI-coworkers and do amazing things with fewer people" — echoes "New Economy" to me and paging Brian Merchant...
- (30:20) Defines AGI as a system that "could do better at making decisions than anyone we could possible have be like say president of the United States" (Um, that is an interesting benchmark these days, but let's not digress)
- (33:22) "the jobs 100 years of from now look nothing like the jobs of today, nor do that many of the jobs of today look like the jobs of a few hundred years ago.". This one is really interesting, because that argument sounds pretty convincing until you notice he is talking about a timescale of hundreds of years...
- (36:56) "We are burning cash very fast. It is not a serious situation [...] if we can spend a billion dollars this year to make $3 billion next year, there's a lot of capital in the world that wants to do that. So we will raise a lot of money, equity and debt" (IPO coming up, but that was clear already). He's promising 200% return on investors in a year.
The actual statement on a new architecture is an answer to a question "What AI subfield today feels like OpenAI in 2016, a little undervalued or underestimated but could shoot out of nowhere?" Then he answers 'from the research perspective': "I bet there is another new architecture to find that is going to be like a big gain as transformers were over LSTM. And I think you finally have models that are smart enough to help do that kind of thing". So we are already close to AGI and that will speed up the improvement of AGI even further.
The full video is here: https://www.youtube.com/watch?v=FjlymGBt-vY
Basically it is the same as always, no real change. The beginning was also interesting. Sam is a believer in this, he has been smitten by the improvements from transformers and he has translated that rapid curve up into an exponential curve, not the beginning of the S-curve which it almost certainly is.
THREE THINGS: First, thank you for the outline--very helpful, though I want to view the video also.
SECOND, about: <- (22:44) "Science will be automated" (in the coming years)> There is a "catch" here that needs unraveling. That is, does Altman mean the science of the machine/ artificial part of intelligence, e.g., the physical sciences of it? OR does he refer to the human sciences that have an enormous influence, not on the physics but on the content?
I have written here before that it seems to me that the narratives confuse the natural elements of humans (our physics, biology, etc.) with the now much more mature social and other sciences (and the arts and history, philosophy, theology) all of which move "up" from the natural/physical to the normative that relates, (basically) to what (ironically as natural) is consciously "agentic" about human beings. And so the question becomes, what exactly informs and provides the basic dynamics of the albeit-modified freedom of thought, deliberate choices, and action that human beings regularly exhibit?
Or to put it more simply, one does not develop emotionally, intellectually, socially, ethically, politically, or spiritually by merely hearing sound (rather than language), putting a bandage on a cut or, eating vegetables rather than dirt.
And THIRD< about: <Then he answers 'from the research perspective': "I bet there is another new architecture to find that is going to be like a big gain as transformers were over LSTM. And I think you finally have models that are smart enough to help do that kind of thing". >
Does he mean only ONE architectural element? or are there perhaps many that, first, develop differently in history, and that do not hail back only to the physical/natural sciences as their mode, but to massive elements that concern human normativity?
I don’t think Sam is that thorough in his thinking, it vvseems all very shallow and fuzzy to me. I don’t think people should pay too much attention to what he says.
You’re speaking of someone who said physics will be solved, I doubt he thinks as deeply as you do on the matter.
Altman does not understand much of anything. One should understand him as an embodiment of the cult of mediocrity that dominates much of the world at the moment. Altman does not have a degree in physics or in any other kind of natural or social science; rather, if I recall correctly, he studied undergraduate computer science, dropping out after two years because he was too cool for school and wanted to play startup with his parents' money. Like the other members of this cult of mediocrity, he profits from work done by people who are, in many cases, far from mediocre, and what little knowledge he himself has is secondhand at best. When he says something, it should be understood as somewhere between a lie, a confident assertion of his ignorance, or in the best case, a distorted recollection of what someone actually knowledgeable working at the company told him.
This is nothing more than a case in point. Regardless of whether current transformer models are capable of doing everything that scientists can, most physicists believe that one of the largest obstacles to developing a complete theory of physics is that there are interactions that occur so rarely, so quickly, at such small resolutions, or at such high energies (and many of these things are related), that either a great deal of time, extremely large measurement devices, or both, would be needed to distinguish between competing theories that are equally consistent with current evidence. Perhaps chatbots could come up with some very clever idea to avoid these limitations, but perhaps not. Unless the clever idea is taking over the world and building a particle accelerator out to the orbit of Jupiter, which is something I do think Altman and his crew might be foolish enough to do by accident.
Lex Ovi: If that's a compliment, I'll take it.
However, it's hardly "deep thinking" to realize the difference between
(a) physics/nature and their empirically established and replicate laws and protocols as guides for platform building, and the folly of
(b) applying them directly to human content where human beings and their normativity is what informs much of what is scaled, and that, though
(c) human content albeit standing on sentient experience governed by physics and nature in the time-space continuum, still raises up to different levels and gradients to define even past and future speculations about what we mean by wholly human experience and reality at different moments in human developmental patterns.
"AGI itself will help to find *another* big breakthrough for (that) AGI." ... So in order to have AGI we first need AGI to help us invent it? Cool, cool, cool.
No, he doesn't say anything about kinds of breakthroughs further down. Might be supercheap AGI for instance, might be anther improvement. The statement isn't about *reaching* AGI at all, that, he says, is going to happen in the coming two years on the current transformer trajectory.
He is just recycling I. J. Good's increasing machine capability as each more powerful iteration is built by the previous one, heading towards the singularity. Of course, he might just have read The Hitchhiker's Guide to the Galaxy books and recalled that to answer the question of "What is the answer to the ultimate question", the supercomputer Deep Thought designed the next more powerful computer...the Earth.
The tell isn't the admission of uncertainty—it's the continued spending. Altman can say 'we need new architectures' and mean it, while still building 00B data centers for the current one. Institutional momentum doesn't reverse on honest reappraisals; it just absorbs them. The real concession would be a change in what gets funded, not a change in what gets said.
Well, maybe the dawning realisation that OpenAI and Anthropic will not be able to meet their financial obligations will get through the poisonous AI-hype-fog much quicker than any other reconsideration. Being hit in the purse hurts the most...
Nicely put, in vintage ChatGPT style :)
The conversation about AGI obscures the real danger of LLMs and massive data centers. The LLMs are extraordinarily good at sifting through massive amounts of data. Essentially they are search engines on steroids. They are very useful tools in the hands of individuals using them for benign or good purposes. They are very dangerous tools in the hands of power seeking individuals.
Even with exponential growth in hardware processing, traditional computing methods and software it was really not possible to do mass surveillance of a population of 330+ million people. The LLMs (aka AIs) have changed that. You are no longer anonymous because you were one person in vast mass of data. The LLMs can find a needle in a massive haystack in seconds.
Based on some back of the envelope calculations, it appears that a typical Gemini cluster could process in real time the combined voice and text output from roughly 250,000 people. Google owns roughly enough compute infrastructure to host several hundred Gemini clusters. I don't know anything about what the other companies have but the LLMs and data centers in existence today would have little difficulty identifying and then tracking in real time the text and voice output of the 3 to 10 million people most likely to be active resisters of an authoritarian government.
I think the real danger of "AI" is not an AGI that will destroy humanity. It's surveillance and data processing (aka LLMs) technology in the hands of an authoritarian government who can use the force of government to control people who would most actively resist an authoritarian takeover like what we're seeing today.
Control of masses of people ultimately comes down to the physical force of government applied to people resisting that government who can be identified and tracked by that government. We've seen this administration centralize data, extract that data, support the building of large scale data centers for use by government to process that data, fund a national paramilitary force, and fund defacto concentration camps.
I don't think this government cares about creating an AGI - it has what it wants right now - the ability to identify and control.
It's starting to feel a lot like we are starting to live the 2011 TV series Person of Interest, which was science fiction then. Now, it's not far from the technical reality of 2026.
Jim Cook: Don't forget the leverages for getting "there," e.g., extortion, bribery, threats to one's family (apparently used during the first and second impeachments) and the old standby: high-rise windows.
Your assumption about how good the current LLM models is close but not quite there. Models do inference. Search + RAG are helpful I. Fimdong source material. However, a recent Stanford AI research (published in the Journal of Empirical Legal Studies) paper show how this fails as more data is added the the accuracy of relevant sources (documents) declined at scale. It was consistent and repeatable. Still much room to improve.
Thanks for the reference to the paper. I found it at this link: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf
Did a quick scan but have to go back and read it more carefully.
I agree about much room to improve. I've been using the cheapest non-free version of Gemini as an augmentation to basic web searching. I've noticed multiple instances of it making up links and getting fairly basic information wrong. When corrected, it "apologizes" and usually provides correct information. However, I've found a few instances where it continues to provide the same incorrect response no matter how much I correct it. My overall take is it's a useful tool (generally saves me time over a simple search query) but you do have to verify information presented. I've also had some success with providing explicit instructions to verify every link, never give me an invalid link and provide something like "<Link not found/available>" instead.
These are just tools, not "intelligences". You outsource your intelligence to a tool at your own risk. I'm a retired attorney and would not use any citation from a LLM without verifying the citation and that the case actually supported the position I was advocating. New attorneys just using the output without verifying are putting their careers at risk, not to mention the harm to the clients they supposedly represent.
I don't think the errors are much of a concern to an authoritarian government using the tools for surveillance/control. Errors causing misidentification, etc. (See: https://www.businessinsider.com/flock-safety-alpr-cameras-misreads-2026-3) may actually be a feature rather than a bug because reports of the negative impacts are likely to make people more concerned about how resistance could harm them. Basically the goal is suppression and if a few people get hurt along the way, the people in power are OK with that.
And exactly where does one go to present/develop/refine/discuss such a new architecture?! Places like NeurIPS and ICLR seem stuck in looking for statistical approaches ONLY. Exeter Uni’s SiC 2025 is so far the only place I’ve found that seems open, but also seems poorly equipped (intellectually) for the task: https://youtu.be/eTLMu_CEa2E?is=udraKihpC4dVeOel
With 1/5 of the worlds hyrdocarbons vanishing overnight, good luck finding enough power to scale anyways.
The politicians and tech bros who so desperately want AGI as fast as possible dont want AGI per say, they want control. The evidence is more and more clear. This is a "people problem" at 10x scale.
Ultimately, the more people wake up to the reality of this, and the more people realize the marketing hype and fear is a lie, the faster we can get out of this nightmare.
AI was here, is here, and will continue to be here.
But that doesnt mean we have to blindly accept enshitification of everything because of it.
There has been so much hype, so many lies, and so many affirmations from these characters that I feel they suffer from a short memory and a broken record syndrome. Perhaps they have become what their models represent? Indeed, new architectures are required, but we also require new people outside this realm of hallucinated fantasy. We need responsible, accountable, and truthful people. He, and many others, do not seem to belong to this category.
Slippery Sam’s little escape clause — effectively, that current AI will find that new architecture — is the last — rather contradictory — line of defence.
I wonder if this is part of setting the stage for the IPO, so the current shareholders would like to recoup as much as they sill can by fleecing the retail investors. Claiming they are near to AGi would probably be legally problematic, so this way they still sort of can.
Turns out when you watch the whole thing (see https://garymarcus.substack.com/p/breaking-sam-altman-concedes-that/comment/228596914) this statement is wrong.
>It’s past time to look for new architectures
Or maybe, just maybe, we could stop digging the hole deeper and wash our hands of this whole sorry project. It's past time for us to wake up from this mad, bad, dystopian feverdream.
I am ok with no AGI for now. They do useful things. The real question is do they do enough to justify the costs and investments? Probably that’s a “sort of”…
GPUs are also good at simulations (classic stuff, not AI).
If we can repropose the ones already built for more generic workloads and cut down the cost for 3D simulation and optimization of various objects we use and need (FEM, geometric optimization, global optimization) then the money won't be wasted.
Just need to think different.
Hm, I don't think it is that easy. Yes, GPUs are used in simulations. The problem might be the accuracy and resolution of the data types the GPUs are optimized for. So the type of data AI GPUs are optimized and built for might not be suitable, i.e. accurate enough for weather or nuclear simulations.
Yeah, that is a problem. But the GPU architecture structure can be changed in the data center to accommodate different workloads.
LLMs will not go anywhere. They are overhyped but still useful if used carefully in specific use cases where verification of the output is possible to be done automatically. Code generation is such a domain.
There is stll the problem of the extreme inefficiency from energy perspective but it can be mitigated at least partially by using dedicated hardware.
Interesting enough, but in many SciFi movies the “advanced” computing is based on crystals. Now, here's the thing, if you wish to optimize a trained ANN you can “set it in stone” by building the neural net including the weights in a “chip tower” that can be fancifully shaped as a large crystal in order to also disipate energy.
So, SciFi is Sci because it has less “Fi” and usually can be realized in a future ahead of us.
Again, changing the GPU architecture is not going to be easy or cheap, as you would have to replace the whole racks with their bespoke cooling and power provisioning. This would not be like replacing ordinary servers with a different model.
Also, actually running the GPUs needs tasks where such costs are not relevant. Nuclear simulation, of course, but *normal* tasks would have to be chosen very carefully to ensure that the costs are way below of actual value created.
Spare some RAM guv’ner? I haven’t had a multitasking session for a week…
The world is begging for upgrades, but all we can get are the Windows kind.
Even if LLMs plateaued, right here, today, it would probably become transformative. That’s why they need the data centers.
So is AI slop worth contributing to the Climate Crisis? I vote NO!