The hope would be that machine, learning, data, science, and statistics experts who dismissed deep learning and generative AI as legitimate paths towards AGI are remembered. I’ve worked all over big tech and I’ve never found utility for deep learning in the work that I did.
It’s laughable in any case because it doesn’t require three weeks of electricity for New York City to teach a toddler language. They learn with almost no examples at all, while generative AI requires billions.
I’m looking forward to blogging about this in the next week or so. Thanks Gary for everything you do.
A big misunderstanding: LLMs aren’t *language* models, they are *token* models. Unimaginably large token models that can from pure ‘volume’ approximate linguistically correct text almost perfectly without having language, and that can — less perfectly, though — approximate text that would result from understanding without having understanding
Correct, and that's their problem. They don't reason, think, or understand, and perhaps worse, they don't know the limitations of their knowledge, so you have a massive case of Dunning-Kruger, biased in favor of false negatives. I think there are good use cases for them, for example in answering questions that have already been authoritatively and repeatedly answered. Coding advice would be a good example.
Agreed, except: coding is one of the uses that are probably no use beyond basic advice for novice or irregular users of a coding language. The big issue is that code (IT in general) is fundamentally extremely brittle (being discrete logic) and thus extraordinarily vulnerable to the errors that LLMs cannot prevent to make (grammar, yes, meaning, no).
LLMs are a way to readily (quickly and mindlessly) produce “kinda-sorta-works-but-has-lots-of-bugs-and security-flaws” software which, not incidentally, is undoubtedly why Microsoft execs are so enamored with them, reportedly currently using them to produce 30% of Microsoft’s new code (KSWBHLOBASF software has been Microsoft’s MO and bread and butter since day one.)
Also not incidentally, it’s no surprise that OpenAI LLMs would produce KSWBHLOBASF software, given that they were undoubtedly partly trained on Microsoft’s code base.
It’s an infinite regression of regression (aka, Regression to the Microslop (TM), well below the Mean)
These x% code *generated* can be a misleading number. The number you want is ‘how much of that gets into production’ (and how much is intermediary ‘code-slop’ — there are so many types of AI-slop these days)
They are called Language Models, because they are used to model language. The fact that they tokenize speech is just an implementation choice to make them more efficient.
Personally: nope, but since it is meant to *produce* language, you could say that.
The problem is that token statistics sits on a level below language. It is like ink droplet statistics on a printed page isn't language (or understanding) either (this analogy by the way comes from around 1888 by Frederik van Eeden). It has little to do with the efficiency (even if tokens, etc. are engineered for that)
The statistics of these machines enable an *approximation* of that what would have followed from actual language (grammar, meaning) understanding. You get 'an approximation of the results of understanding, without the understanding'.
So is it a 'language model' (LLM) or an 'approximation of language model' (LAOLM). I think clarity is served by labelling it based on tokens as the other one misleads.
I have no problem with inefficiency up-front as any useful technology might require.
I think you're wrong in this case--generative systems and LLMs are NOT a necessary step in building AGI. It also isn't clear that we'll require trillions of dollars to accomplish AGI (there's no proof, except that LLMs won't accomplish intelligence the way the brain does). There are always alternatives, and with every boom/bust AI summer/AI winter, we've suffered from the desire to overpromise and overcommit. It's dangerous, and like most of the cycles, it costs us as taxpayers and as scientists immensely.
The trillions wasted could have been used to solve much more pressing problems, like water, food, and education. Further, LLMs can't solve CSPs, SAT, or any other NP-complete problem.
If we want AGI, we must begin with the only true example--the human brain. Relying instead on asymptotic behavior of lumbering, inefficient systems to ape narrow tasks leads to the simple problem of sample complexity. I can already solve SAT with OLS, but it requires a system of exponential size if P!=NP. OLS used to be considered AI.
The brain doesn't rely on gigantic samples to accomplish what it does. Thus, trying to build a brain using wasteful data centers will lead to more waste and no real solution--for instance, image classifiers are easily fooled by changing a few pixels. Humans don't see the difference, but the classifier gets the wrong answer. What has this taught us? The brain isn't using deep learning to classify images. Did this dissuade DL enthusiasts? Not one bit. Instead, there's a call for more money.
The grifters in tech would want investors and workers to believe a fantasy. They sell you a Ferrari for 20% the cost, but you quickly discover after paying that you were looking at an incredibly detailed hologram. It won't be sustainable.
Appreciated. It's heresy in some circles (and I'm not doing my career any favors by criticizing it), but we can all support one another when taking unpopular stances.
It is, in fact, money American taxpayers pay which ensures the inefficient up-front research can happen. My grandparents paid taxes leading to the creation of the computer, the internet, integrated circuits, and much more. None of the current infrastructure would exist without tremendous subsidies from the state sector. This continues to be the case--there is almost no dynamic part of the economy which can survive without government intervention. But even if I took what you said at face value (people can waste their money if they like), there's a difference between twirling our mustaches as we knowingly stick our hands in their pockets and blind faith about the outcome. Neither is befitting of an expert--especially if we're slurping up government money to make it happen.
Frankly, the case has NOT been made that LLMs will net trillions of dollars. In fact, there's plenty of evidence to the contrary. I know it's tempting to believe that the rivers of invested cash will continue, and I'd probably be better off financially in the short-term supporting it. But it's worse than an illusion--it's a scam, designed to trick non-experts into spending their own time and money improving it.
I'm reminded of Theranos and the Cruise when considering the trickery--lie about the system's capabilities by mocking them out. Yes, a pianist learning over a lifetime represents a significant cognitive feat. But even then, any unit of interest (say minutes practicing) can't number more than the very low millions. And unlike the case of training of an LLM, the brain can't reconstruct those minutes. The LLM by contrast requires this for probable sequences. As a pianist myself, I can't be asked exactly which five notes I played twenty years ago in a single practice session. The point is the brain accomplishes immense feats without relying on the infrastructure of a data center. Or even the laptop I have in front of me.
My comparison in no way undervalues human cognition. By contrast, I am saying that LLMs are highly inefficient, wasteful, and error-prone as a surrogate. It's literally like insisting that accomplishing nanotech requires we build 100T hammers.
The problem here is classic path dependency--something seems to work well in a few spots, so we invest more and more time and resources into it with the unswerving faith it can accomplish it. Eventually, we have to admit that we wasted years debating a technology which would have hit peak returns in the distant past.
It's perhaps worse than ordinary enshittification.
I don't think you have any basis to claim to know what @cnpslagle does or does not know or what they estimate.
Just from the comment posts they have provided here, I would say that there is scant evidence they are underestimating "the time, effort money or risk in getting from prototpes to commercially useful products": if anything, I'd infer they have a reasonable grasp on such things.
It's perhaps ironic that this is one of the key problem areas with LLM/GPT-based systems as a basis or central core of an AI system. They neither "know" anything, nor are they capable of "knowing" what others know. There is no thought / no "knowing" - at least not to the extent we understand it.
You beat me to it. One of the hallmarks of the 1920s leading into the stock market crash was the enormous surplus of radios, clothing items, toys, and more in warehouses and factories. The folks working there couldn't afford the things they made. Investors thought demand would continue to grow.
They undercut the intended consumers, and thus, it collapsed.
Replacing everyone except the top permille means no one is left to buy a model 2025 ChatGPT, and no data center thus will remain open.
But I don't think it'll advance that far--LLMs can't really execute on tasks very well. They rely on existing token order to answer and choose. While there is a cloak of no regulation, the tech will spill into many facets. But people aren't so stupid as to believe it when they're fleeced by the organizations promoting the tech. If writing a suicide note for a teen isn't enough to slow this advance, I don't know what is.
The grift is to soak up all the resources while persuading others (the government, middle classers, other investors) to put themselves on the line.
GenAI presents actual scientific and technological progress. The current (and near future) capabilities are though being heavily inflated and oversold. The worst part of the current situation is that concentration of funding to LLMs and diffusion models starves other needed investments. I’m skeptical about the possibility (and frankly also utility) of AGI, but current GenAI already has valid use cases. Viable business models? Not so much.
It's fair to say that technological effort has been expended, however I'm struggling to see - some 3 years or so into the ballooning hype bubble - what *progress* has been made in a real, practical, tangible sense.
Insiders who'd been deeply involved in the preceeding years before the media hype could already see the coming technological plateau / end-game for LLM's / GPT's - certainly in and of themselves as a pathway to anything revolutionary in progress towards generally useful AI systems.
When Chat GPT burst into the media spotlight, those insiders warned us of the coming plateau: now here we are with those warnings vindicated many times over.
Staying constrained within the LLM/GPT-space and pouring endless investment into diminishing cycles of lint polishing neither makes my pocket more useful, nor does it give me - or us collectively - any real forward technological progress.
Fanboys can love and admire that LLM/GPT-hammer as much as they want, but redefining every problem they clearly can't solve as a "nail" doesn't make the hammer more useful.
LLMs got so much funding because investors believe that by scaling them up they can deliver much more than they currently do. That is now pretty much proven to not be the case. All of the current viable LLM use cases (like code completion, report generation, summarization, image/video generation, …) can hardly provide any meaningful return on their investments. The training costs are simply too high compared to what users are willing to pay for LLM output. Investors though still seem to believe that AGI (whatever that actually means) is right around the corner and will solve all our problems :).
Meanwhile, look at the stress to the existing power grid, rising electricity costs being passed on to consumers, and the threats to our water supply and environmental chain. If it's a dystopian hellscape one wants to live in, then we're definitely on the right path.
All so that people can churn out AI slop on YouTube, TikTok and Facebook.
Neuroscience literature indicates that the adult human brain typically uses about 12–20 watts of power, which is equivalent to the electricity needed to power a small LED bulb or a computer monitor in sleep mode. During wakefulness and normal activities
Ahhg! Can't find it. but really amusingly, googl's ai keeps offer up entirely different descriptions of cartoons wit the same caption. Giant eyes, cannibals, axe wielding wives, men sprouting vegetables.. Just add an extra space or letter(or remove same) and disorder is revealed
One important qualification that effects your message: Softbank’s sale is to take profit (and there currently is a lot of that for some investors in NVDA) to invest in more AI ventures that are startups and juniors.
Thank you for sharing these. I am really getting sick of the Generative AI as "God" hype that is going on. The NYT Opinion just had someone write something on this. What makes me sick is that there is no clear definition of what the writer means by the word "God." As we know from history it has meant many things and still does today depending on one's religious beliefs (including not having any). And it steps over the fact that humans have been redefining the one true God for centuries (too many books on this, but Karen Armstrong's, "A History of God," is a good place to start). So, the whole argument about creating "God" is baseless. The idea that "God" is some kind of super-intelligent being above human intelligence, is just one (and a weak one at that) of defining "God." Some religions define God as the creator of the Universe! Certainly, that would be lot more than just super-intelligent as it would require the power to arrange all forms of matter (including dark matter). This kind of AI hype seems to be serving some purpose of motivating an unaware public to support pro-development AI policy choices based on OMG thinking. What a scam!
CoreWeave is playing the important-for-American-defense card. That is why it is not zero, yet. It is angling for a bailout, a la Intel.
All AI is now intertwined with Trump. If Trump is impeached or sidelined in the next few months, their hopes of a bail out are done. GOP is getting to the point where it has to decide.
SoftBank is getting out of shovel business and moving closer to the gold mine, as one commentator on X put it. This is a sign that they think better days are ahead.
Google's GPT paper first came out in 2017. ChatGPT 3.5 came in 2022. At the time (and still to some extent at present) belief is scale = AGI. Quadrillion parameter models running on infinite power from 100 nuclear plants will lead to AGI.
Theoretically probably I guess but diminishing probability given that scale has increased several orders of magnitude since 2022 but intelligence has not. These are clever tools and useful in narrow use cases but the hallucinations are egregiously bad too. Will take several step function improvements to get to AGI and while big tech is betting that $100M paychecks will result in a breakthroughs, fundamental research that leads to breakthroughs takes years, not months. Along the way there will be bodies and winters.
Tech billionaire Peter Thiel's hedge fund has sold off its entire stake in Nvidia (NVDA.O) during the third quarter, a regulatory filing showed, intensifying worries of an artificial intelligence bubble.
The fund, Thiel Macro, sold around 537,742 shares in the AI chip frontrunner in the quarter, the filing showed on Friday. The stake would have been worth around $100 million, as of the company's closing price on September 30.
Inspired by your work, and fed up with AI slop, I just published my first Substack piece on the hidden reality behind the LLM hype within organizations, “workslop”. One’s Agentic (LLM) Efficiency is Another’s AI Slop.
I see several key limitations of LLMs as compared to human intelligence.
-> energy intensity (as one of the replies point it out, we use very little energy, which is about 20% to 30% of daily calorie intake). People don't eat 2x or 3x as much to do 2x or 3x amount of thinking/mental work. The compute/energy intensive nature of LLMs implies that it's a very crude mimic of human intelligence. Neuromorphic chips using spiking neural networks seem to be in a much better position to fundamentally tackle this issue.
-> Humans learn via a "model/theory" based approach, which means that we can form a functional model in our heads with limited sample data, and we can learn from theories directly. Although this isn't perfect ("biases" is a by-product ) this is highly efficient. In scientific fields, it also means that if a person learns a theory well, s/he can solve all the problems that can be solved by the theory. This capability for abstract thinking and translate back and forth between theory and sample data isn't there with LLMs. LLM relies on very large quantity of sample data to "learn".
-> we "update" our mental model as new information comes in when we carry out tasks - training and inferencing happen at the same time - it's interactive. LLMs doesn't have this interactivity. It can "mimic" it using RAG, but it's not interactive. This may explain why longer sessions with LLMs create "drifts" and worse results, while with humans it's the opposite.
-> Repetitive practices can make us super efficient in executing a task ("subconscious" competency) with lower and lower amount of effort (i.e. low energy). Examples include performing instruments, driving, or doing mental math. I don't think that LLMs become more efficient when answering similar questions over and over again. Same level of computational intensity.
LLM is great technological advancement. But it's so new... human intelligence is the result of millions of years of evolution... to put so much hype on something so new is like asking a baby to be a superman right away by force feeding it every day. Of course, the hype itself sustains the money flow, and that should explain it all. Someday we'll have close to AGI, it can be decades, which is already very fast as compared to the pace of natural evolution. What's for sure is that it won't be from the current LLMs "scaling".
Even more critically, human brain follows a "use it or lose it" model, as a result of natural selection. Thus, when we just let these energy suckers do jobs that were there to train our young and keep our own minds sharp, what are we doing to ourselves? I can see that AI/LLM being put for really good uses as our "extensions" - drug discovery, climate simulation, having drones/robots do hazardous tasks, etc. But not "replacement" just to save money, unless those being replaced can find other productive outlets. Many current AI "use cases" are self-harm as a specie. For example, the rampant use of AI by students are keeping them from really learning/absorbing/thinking (how to use AI is critical; what I saw how students are using it are mostly harmful... and their teachers are in no position to truly guide the students in terms of how to use AI)... as a society, we truly do not value our collective wellbeing as tech big-bros succeed in sucking in the money.
You’re pointing to something important. There’s a lot of noise around “signs of trouble” in AI right now, but the real story is quieter. AI evolves in full view, yet the definition of intelligence keeps shifting underneath us. Every time a model masters something once thought to require a mind, we move the boundary and redefine intelligence as whatever machines can’t do yet. These market jolts matter, but the deeper trajectory doesn’t come from the loud events. It comes from the slow, structural forces that are steadily rewriting what we mean by intelligence in the first place.
The hope would be that machine, learning, data, science, and statistics experts who dismissed deep learning and generative AI as legitimate paths towards AGI are remembered. I’ve worked all over big tech and I’ve never found utility for deep learning in the work that I did.
It’s laughable in any case because it doesn’t require three weeks of electricity for New York City to teach a toddler language. They learn with almost no examples at all, while generative AI requires billions.
I’m looking forward to blogging about this in the next week or so. Thanks Gary for everything you do.
A big misunderstanding: LLMs aren’t *language* models, they are *token* models. Unimaginably large token models that can from pure ‘volume’ approximate linguistically correct text almost perfectly without having language, and that can — less perfectly, though — approximate text that would result from understanding without having understanding
Correct, and that's their problem. They don't reason, think, or understand, and perhaps worse, they don't know the limitations of their knowledge, so you have a massive case of Dunning-Kruger, biased in favor of false negatives. I think there are good use cases for them, for example in answering questions that have already been authoritatively and repeatedly answered. Coding advice would be a good example.
Agreed, except: coding is one of the uses that are probably no use beyond basic advice for novice or irregular users of a coding language. The big issue is that code (IT in general) is fundamentally extremely brittle (being discrete logic) and thus extraordinarily vulnerable to the errors that LLMs cannot prevent to make (grammar, yes, meaning, no).
Agree, kinda like an easier way to search Stack Overflow, but there is some advantage in that.
LLMs are a way to readily (quickly and mindlessly) produce “kinda-sorta-works-but-has-lots-of-bugs-and security-flaws” software which, not incidentally, is undoubtedly why Microsoft execs are so enamored with them, reportedly currently using them to produce 30% of Microsoft’s new code (KSWBHLOBASF software has been Microsoft’s MO and bread and butter since day one.)
Also not incidentally, it’s no surprise that OpenAI LLMs would produce KSWBHLOBASF software, given that they were undoubtedly partly trained on Microsoft’s code base.
It’s an infinite regression of regression (aka, Regression to the Microslop (TM), well below the Mean)
These x% code *generated* can be a misleading number. The number you want is ‘how much of that gets into production’ (and how much is intermediary ‘code-slop’ — there are so many types of AI-slop these days)
I’d guess that’s a pretty high % with Microsoft (95% of 30%?)
And, of course, that leaves the remaining Microslop that is not generated by LLMs.
I thought they were “Large Languish Models”
They are called Language Models, because they are used to model language. The fact that they tokenize speech is just an implementation choice to make them more efficient.
Personally: nope, but since it is meant to *produce* language, you could say that.
The problem is that token statistics sits on a level below language. It is like ink droplet statistics on a printed page isn't language (or understanding) either (this analogy by the way comes from around 1888 by Frederik van Eeden). It has little to do with the efficiency (even if tokens, etc. are engineered for that)
The statistics of these machines enable an *approximation* of that what would have followed from actual language (grammar, meaning) understanding. You get 'an approximation of the results of understanding, without the understanding'.
So is it a 'language model' (LLM) or an 'approximation of language model' (LAOLM). I think clarity is served by labelling it based on tokens as the other one misleads.
Laugh Outlandishly Loud (LOL) Models
Hard to imagine something even less efficient than an LLM
https://m.economictimes.com/news/international/us/is-the-us-facing-a-massive-electricity-shortage-heres-the-reason-and-why-americans-should-take-note/articleshow/125391655.cms
I have no problem with inefficiency up-front as any useful technology might require.
I think you're wrong in this case--generative systems and LLMs are NOT a necessary step in building AGI. It also isn't clear that we'll require trillions of dollars to accomplish AGI (there's no proof, except that LLMs won't accomplish intelligence the way the brain does). There are always alternatives, and with every boom/bust AI summer/AI winter, we've suffered from the desire to overpromise and overcommit. It's dangerous, and like most of the cycles, it costs us as taxpayers and as scientists immensely.
The trillions wasted could have been used to solve much more pressing problems, like water, food, and education. Further, LLMs can't solve CSPs, SAT, or any other NP-complete problem.
If we want AGI, we must begin with the only true example--the human brain. Relying instead on asymptotic behavior of lumbering, inefficient systems to ape narrow tasks leads to the simple problem of sample complexity. I can already solve SAT with OLS, but it requires a system of exponential size if P!=NP. OLS used to be considered AI.
The brain doesn't rely on gigantic samples to accomplish what it does. Thus, trying to build a brain using wasteful data centers will lead to more waste and no real solution--for instance, image classifiers are easily fooled by changing a few pixels. Humans don't see the difference, but the classifier gets the wrong answer. What has this taught us? The brain isn't using deep learning to classify images. Did this dissuade DL enthusiasts? Not one bit. Instead, there's a call for more money.
The grifters in tech would want investors and workers to believe a fantasy. They sell you a Ferrari for 20% the cost, but you quickly discover after paying that you were looking at an incredibly detailed hologram. It won't be sustainable.
Excellent, focused, summary critique that cuts through a lot of the hype and hoopla from this industry, its investors and fanboys.
On the money. If I could upvote this multiple times, I woud.
Appreciated. It's heresy in some circles (and I'm not doing my career any favors by criticizing it), but we can all support one another when taking unpopular stances.
I plan to write more on this eventually.
“H-AI-resey”
Given all the hype
H-AI-resey is good
Valley boyz will gripe
Claim “misunderstood”
That is true to type
Just like Sammys should
It is, in fact, money American taxpayers pay which ensures the inefficient up-front research can happen. My grandparents paid taxes leading to the creation of the computer, the internet, integrated circuits, and much more. None of the current infrastructure would exist without tremendous subsidies from the state sector. This continues to be the case--there is almost no dynamic part of the economy which can survive without government intervention. But even if I took what you said at face value (people can waste their money if they like), there's a difference between twirling our mustaches as we knowingly stick our hands in their pockets and blind faith about the outcome. Neither is befitting of an expert--especially if we're slurping up government money to make it happen.
Frankly, the case has NOT been made that LLMs will net trillions of dollars. In fact, there's plenty of evidence to the contrary. I know it's tempting to believe that the rivers of invested cash will continue, and I'd probably be better off financially in the short-term supporting it. But it's worse than an illusion--it's a scam, designed to trick non-experts into spending their own time and money improving it.
I'm reminded of Theranos and the Cruise when considering the trickery--lie about the system's capabilities by mocking them out. Yes, a pianist learning over a lifetime represents a significant cognitive feat. But even then, any unit of interest (say minutes practicing) can't number more than the very low millions. And unlike the case of training of an LLM, the brain can't reconstruct those minutes. The LLM by contrast requires this for probable sequences. As a pianist myself, I can't be asked exactly which five notes I played twenty years ago in a single practice session. The point is the brain accomplishes immense feats without relying on the infrastructure of a data center. Or even the laptop I have in front of me.
My comparison in no way undervalues human cognition. By contrast, I am saying that LLMs are highly inefficient, wasteful, and error-prone as a surrogate. It's literally like insisting that accomplishing nanotech requires we build 100T hammers.
The problem here is classic path dependency--something seems to work well in a few spots, so we invest more and more time and resources into it with the unswerving faith it can accomplish it. Eventually, we have to admit that we wasted years debating a technology which would have hit peak returns in the distant past.
It's perhaps worse than ordinary enshittification.
I don't think you have any basis to claim to know what @cnpslagle does or does not know or what they estimate.
Just from the comment posts they have provided here, I would say that there is scant evidence they are underestimating "the time, effort money or risk in getting from prototpes to commercially useful products": if anything, I'd infer they have a reasonable grasp on such things.
It's perhaps ironic that this is one of the key problem areas with LLM/GPT-based systems as a basis or central core of an AI system. They neither "know" anything, nor are they capable of "knowing" what others know. There is no thought / no "knowing" - at least not to the extent we understand it.
If you fire all those people, consumer spending will go downhill. Hard to make money in a depression
Exactly. It's like burning down your house so you don't have to heat it.
Good idea.
You beat me to it. One of the hallmarks of the 1920s leading into the stock market crash was the enormous surplus of radios, clothing items, toys, and more in warehouses and factories. The folks working there couldn't afford the things they made. Investors thought demand would continue to grow.
They undercut the intended consumers, and thus, it collapsed.
Replacing everyone except the top permille means no one is left to buy a model 2025 ChatGPT, and no data center thus will remain open.
But I don't think it'll advance that far--LLMs can't really execute on tasks very well. They rely on existing token order to answer and choose. While there is a cloak of no regulation, the tech will spill into many facets. But people aren't so stupid as to believe it when they're fleeced by the organizations promoting the tech. If writing a suicide note for a teen isn't enough to slow this advance, I don't know what is.
The grift is to soak up all the resources while persuading others (the government, middle classers, other investors) to put themselves on the line.
It’s just AI-conomics (Artificial imaginary economics)
None of if makes any sense.
The generative AI prophets already told us they had made an intelligence beyond ours and we saw how that worked out.
Now they're recycling the same con under the name of AGI and you want to advocate for them?
How about these geniuses figure out a business model that makes a profit before begging us for a few trillion more?
GenAI presents actual scientific and technological progress. The current (and near future) capabilities are though being heavily inflated and oversold. The worst part of the current situation is that concentration of funding to LLMs and diffusion models starves other needed investments. I’m skeptical about the possibility (and frankly also utility) of AGI, but current GenAI already has valid use cases. Viable business models? Not so much.
It's fair to say that technological effort has been expended, however I'm struggling to see - some 3 years or so into the ballooning hype bubble - what *progress* has been made in a real, practical, tangible sense.
Insiders who'd been deeply involved in the preceeding years before the media hype could already see the coming technological plateau / end-game for LLM's / GPT's - certainly in and of themselves as a pathway to anything revolutionary in progress towards generally useful AI systems.
When Chat GPT burst into the media spotlight, those insiders warned us of the coming plateau: now here we are with those warnings vindicated many times over.
Staying constrained within the LLM/GPT-space and pouring endless investment into diminishing cycles of lint polishing neither makes my pocket more useful, nor does it give me - or us collectively - any real forward technological progress.
Fanboys can love and admire that LLM/GPT-hammer as much as they want, but redefining every problem they clearly can't solve as a "nail" doesn't make the hammer more useful.
LLMs got so much funding because investors believe that by scaling them up they can deliver much more than they currently do. That is now pretty much proven to not be the case. All of the current viable LLM use cases (like code completion, report generation, summarization, image/video generation, …) can hardly provide any meaningful return on their investments. The training costs are simply too high compared to what users are willing to pay for LLM output. Investors though still seem to believe that AGI (whatever that actually means) is right around the corner and will solve all our problems :).
“it can do what other methods cannot“
Induce psychosis and suicidal thoughts, for example.
(And other important stuff like that)
Yes, but if they're smart enough to engineer AGI a simple thing like a profitable business model ought to be easily within their abilities.
And if it's not ...
The AI companies learned from the bankers that a (very) profitable (to the tune of trillions) business model is one that profits off of US tax payers.
Of course. Privatize the profits and socialize the losses.
It's not.
Meanwhile, look at the stress to the existing power grid, rising electricity costs being passed on to consumers, and the threats to our water supply and environmental chain. If it's a dystopian hellscape one wants to live in, then we're definitely on the right path.
All so that people can churn out AI slop on YouTube, TikTok and Facebook.
Neuroscience literature indicates that the adult human brain typically uses about 12–20 watts of power, which is equivalent to the electricity needed to power a small LED bulb or a computer monitor in sleep mode. During wakefulness and normal activities
Yeah, natural intelligence is still much more energy efficient than any AI. … and it also learns continuously.
Agree. A lot, lot, lot ……more
“Neural nit wit” would be an apt name for an LLM
Check out the "certain indicators than cannot be ignored" New Yorker cartoon by Charles Addams.
Ahhg! Can't find it. but really amusingly, googl's ai keeps offer up entirely different descriptions of cartoons wit the same caption. Giant eyes, cannibals, axe wielding wives, men sprouting vegetables.. Just add an extra space or letter(or remove same) and disorder is revealed
Google: Addams "certain indicators that cannot be ignored" cartoon
The look under 'Images'
Even darker than expected! Good tip.
A sixth ominous sign is Sam Altman’s charred pants
One important qualification that effects your message: Softbank’s sale is to take profit (and there currently is a lot of that for some investors in NVDA) to invest in more AI ventures that are startups and juniors.
SoftBank is investing more in OpenAI.
Clearly you are missing it Gary, 🤣 Sam says he has it all under control. How could that be wrong.😉
Thank you for sharing these. I am really getting sick of the Generative AI as "God" hype that is going on. The NYT Opinion just had someone write something on this. What makes me sick is that there is no clear definition of what the writer means by the word "God." As we know from history it has meant many things and still does today depending on one's religious beliefs (including not having any). And it steps over the fact that humans have been redefining the one true God for centuries (too many books on this, but Karen Armstrong's, "A History of God," is a good place to start). So, the whole argument about creating "God" is baseless. The idea that "God" is some kind of super-intelligent being above human intelligence, is just one (and a weak one at that) of defining "God." Some religions define God as the creator of the Universe! Certainly, that would be lot more than just super-intelligent as it would require the power to arrange all forms of matter (including dark matter). This kind of AI hype seems to be serving some purpose of motivating an unaware public to support pro-development AI policy choices based on OMG thinking. What a scam!
CoreWeave is playing the important-for-American-defense card. That is why it is not zero, yet. It is angling for a bailout, a la Intel.
All AI is now intertwined with Trump. If Trump is impeached or sidelined in the next few months, their hopes of a bail out are done. GOP is getting to the point where it has to decide.
Is SoftBank selling Nvidia a sign if they are only doing it to buy more OpenAI?
SoftBank is getting out of shovel business and moving closer to the gold mine, as one commentator on X put it. This is a sign that they think better days are ahead.
All that glitters is not AGI (Artificial Golden Intelligence)
Fool’s AGI actually looks a lot like real AGI
And of course (ironically) Fool’s AGI is not as soft as the real thing.
Google's GPT paper first came out in 2017. ChatGPT 3.5 came in 2022. At the time (and still to some extent at present) belief is scale = AGI. Quadrillion parameter models running on infinite power from 100 nuclear plants will lead to AGI.
Theoretically probably I guess but diminishing probability given that scale has increased several orders of magnitude since 2022 but intelligence has not. These are clever tools and useful in narrow use cases but the hallucinations are egregiously bad too. Will take several step function improvements to get to AGI and while big tech is betting that $100M paychecks will result in a breakthroughs, fundamental research that leads to breakthroughs takes years, not months. Along the way there will be bodies and winters.
Oh, my bad. Sounds like Marcus then. My apologies.
Tech billionaire Peter Thiel's hedge fund has sold off its entire stake in Nvidia (NVDA.O) during the third quarter, a regulatory filing showed, intensifying worries of an artificial intelligence bubble.
The fund, Thiel Macro, sold around 537,742 shares in the AI chip frontrunner in the quarter, the filing showed on Friday. The stake would have been worth around $100 million, as of the company's closing price on September 30.
Inspired by your work, and fed up with AI slop, I just published my first Substack piece on the hidden reality behind the LLM hype within organizations, “workslop”. One’s Agentic (LLM) Efficiency is Another’s AI Slop.
I see several key limitations of LLMs as compared to human intelligence.
-> energy intensity (as one of the replies point it out, we use very little energy, which is about 20% to 30% of daily calorie intake). People don't eat 2x or 3x as much to do 2x or 3x amount of thinking/mental work. The compute/energy intensive nature of LLMs implies that it's a very crude mimic of human intelligence. Neuromorphic chips using spiking neural networks seem to be in a much better position to fundamentally tackle this issue.
-> Humans learn via a "model/theory" based approach, which means that we can form a functional model in our heads with limited sample data, and we can learn from theories directly. Although this isn't perfect ("biases" is a by-product ) this is highly efficient. In scientific fields, it also means that if a person learns a theory well, s/he can solve all the problems that can be solved by the theory. This capability for abstract thinking and translate back and forth between theory and sample data isn't there with LLMs. LLM relies on very large quantity of sample data to "learn".
-> we "update" our mental model as new information comes in when we carry out tasks - training and inferencing happen at the same time - it's interactive. LLMs doesn't have this interactivity. It can "mimic" it using RAG, but it's not interactive. This may explain why longer sessions with LLMs create "drifts" and worse results, while with humans it's the opposite.
-> Repetitive practices can make us super efficient in executing a task ("subconscious" competency) with lower and lower amount of effort (i.e. low energy). Examples include performing instruments, driving, or doing mental math. I don't think that LLMs become more efficient when answering similar questions over and over again. Same level of computational intensity.
LLM is great technological advancement. But it's so new... human intelligence is the result of millions of years of evolution... to put so much hype on something so new is like asking a baby to be a superman right away by force feeding it every day. Of course, the hype itself sustains the money flow, and that should explain it all. Someday we'll have close to AGI, it can be decades, which is already very fast as compared to the pace of natural evolution. What's for sure is that it won't be from the current LLMs "scaling".
Even more critically, human brain follows a "use it or lose it" model, as a result of natural selection. Thus, when we just let these energy suckers do jobs that were there to train our young and keep our own minds sharp, what are we doing to ourselves? I can see that AI/LLM being put for really good uses as our "extensions" - drug discovery, climate simulation, having drones/robots do hazardous tasks, etc. But not "replacement" just to save money, unless those being replaced can find other productive outlets. Many current AI "use cases" are self-harm as a specie. For example, the rampant use of AI by students are keeping them from really learning/absorbing/thinking (how to use AI is critical; what I saw how students are using it are mostly harmful... and their teachers are in no position to truly guide the students in terms of how to use AI)... as a society, we truly do not value our collective wellbeing as tech big-bros succeed in sucking in the money.
It's feeling kind of bubbly here.
You’re pointing to something important. There’s a lot of noise around “signs of trouble” in AI right now, but the real story is quieter. AI evolves in full view, yet the definition of intelligence keeps shifting underneath us. Every time a model masters something once thought to require a mind, we move the boundary and redefine intelligence as whatever machines can’t do yet. These market jolts matter, but the deeper trajectory doesn’t come from the loud events. It comes from the slow, structural forces that are steadily rewriting what we mean by intelligence in the first place.
James Li had a good video on AI bubble today