In a recent interview, Dario Amodei dismissed proofs that LLM alignment is impossible as “nonsense” and “gobbledygook”, adding that he considers people who have published proofs like this morally and intellectually unserious.
The gall. People like myself have spent the time and hard work publishing work like this in peer reviewed academic journals.
Meanwhile, neither he, nor any other developer—nor anyone on any of their teams—have bothered to engage the proofs, as they continue to post methodologically flawed, non-peer reviewed experimental results on their websites and all across the ArXiv, claiming “breakthroughs” left and right.
They are doing bad science, as you point out, in large part because they don’t have the time or wherewithal to respect or uphold basic, longstanding, proven scientific norms—such as, you know, bothering with peer review or addressing proofs and other critiques that don’t align with their PR and bottom line.
Thank you for sticking to your guns and speaking the truth with genuine expertise. As Churchill said, 'Truth is incontrovertible. Panic may resent it. Ignorance may deride it. Malice may distort it. But there it is.'
This reminds me of Columbus discovering America and never being able to admit it wasn't India. Eventually, folks had to admit that discovering America had its own, lesser value.
How could AI based on the median human reality that is the web [LLM] be substantially more than mediocre?
I don't know why anyone is surprised by the results being produced. This system injests both the best and worst of humanity; the genius and the BS. When you average them together—since the system can't reason or discern—of course one is going to get middling, mediocre results in the output.
Why is it “bullshiting”? It's, surprisingly acute, as I wrote before: in a world of “scale is all you need” AGI makes sense. Because “scale” is more-or-less one parameter, as you increase it you first match human and then surpass it. AGI is when you match human, ASI is when you bypass it.
In a world where “scale is NOT all you need”… AGI is meaningless: because AI achieve different things that human can do at different times “AI that's approximately equal in capabilities to human” simply would never exist, only “AI that's strictly worse than human” and “AI that's strictly better than human”.
Maybe better to say not that this proclamation is “bullshitting”, but more of a heavy emphasis of the term was bullshit: it was based on flawed hypnotises, hypnotises is, now, disproven… term is no longer useful. End of Story.
Reminds me of Trump saying he was being sarcastic when he said he'd end the war in Ukraine within 24 hours of being elected. Wonder how long it will take Altman to get to his version of "I don't think Putin wants to end the war."
I don't think we've hit peak AI, but I do think we've hit peak AI hype cycle. Next stop: the valley of disillusionment, featuring all the companies that have burned billions in this tech.
Nice to hear from Gary about the long overdue change of the public perception. If we are talking about science, my team has been for decades pursuing what I would call SyNe AI (symbolic AI with support from statistics-based — “neural” tools) which is where the real explanatory science should be looking instead even of what today is called neurosymbolic (NeSy) AI which is for most of its practitioners is just sprinkling some hand-crafted knowledge over what still is an LLM substrate, which won’t get us very far scientifically. But this comment belongs to a different discussion, which I hope will become more central in AI — what kind (or kinds) of neurosymbolic symbiosis would be scientifically more promising.
Hi - interesting comment. Do you have some references for the approach? Which I take it would mean an architecture that envisioned both as co-equal components from the start vs Sym as a semantic patch - and a clearly scoped methodological regime for the dataset (vs. scrape the web)?
BTW - I wonder how we will know whether/how many fixes to purported LLM flaws are just symbolic AI kluges hidden under the covers?
There is an obvious other way - Semantic AI - where the language is English, and acts as the transfer mechanism for structure from the head of one person to another, or to a machine. Every word (and the elided words) is represented by an object or an operator representing the particular meaning the word has in context (some words have multiple POS and/or 60-80 meanings). At the moment, we have a vocabulary of 45,000 words, 10,000 wordgroups (a group of words that has a figurative meaning - "behind bars", "a bridge too far").
An insuperable problem for what you are doing is that humans are not good at complex problems (the Four Pieces Limit), whereas a machine can keep all the connections in the network (tens of thousands) in sync while the human fiddles about with a particular detail, staying at or below their limit.
Eh, no. How do you describe meaning? And for what uses? Read some of our stuff. Of course, it’s too much to read, and it would be difficult. But you’ll learn a lot if you try. Cheers!
Lol Gary! Bingo, again. In fact, pre GPT3.5 they humble-bragged about them behaving responsibly by not releasing their hyped up token calculator all in one piece because doing so would be irresponsible - because the LLM was so dangerous. LMAO.
Yes - LLM/GPT-based systems are certainly masterful at understanding the forms and patterns / construction of output that is generally pleasing and / or valuable to humans (to a greater or lesser extent).
As I've noted for a while, they will likely have a useful part to play going forward in input and output construction within the context of a distributed / heterogeneous AI system of services that include one or more domain-specific world-model engines.
They will book-end the interaction by providing a translation from and back to human-likable form.
I was specifically noting that LLM/GPT-based systems - where they have ingested a sufficient quantity of text and images - appear capable of both a) interpreting human-created / human-interpretable text and images and b) constructing human-recognisable and human-likable text and images. They are capable of inferring from an input using a corpus sufficient "lexical semantics" and "visual semantics" to understand the form and logical meaning of text and images, and likewise .
With those capabilities, they could conceptually form a useful function at the boundary of a distributed AI system, where other parts of that system including domain-"grounded knowledge" and contextual awareness work in collaboration to enable the creation of well-formed, meaningful, trustable responses to inquiries.
| they mimic the observable surfaces of objects and behaviors/actions,
| and thereby provide an interface (but not much more)
If you're referring to an Interface Theory of Perception, then perhaps that's the case, though I'm not implying or suggesting that - I'm not familiar enough with that theory to comment.
I was reading a desription of a test tn which the AI program was able to predict the positions of planets, but had no idea of the gravitationall forces actually determining their motion and pisition. It did not infer a real model. The movie "Catch Me if You Can" about the imposter and check fraudster Frank Abagnale (starred Leonardo di Caprio) then popped into my head. He mimicked the appearance and behavior of an airline pilot, for example, with none of the underlying knowledge and skill. LLMs have been characterized as supercharged auto complete, which is consistent with mimicry. We mammals do learm from mimicry, but only so much. As a small boy, I saw contractors installing a new furnace in a large hole in our kitchen floor. Days later, my Mother stopped me from attacking the.linoleum with a knife, "putting in a furnace".
While the progress of the last few years may seem to have slowed, it's more accurate to say that the field is maturing and diversifying. The focus is shifting from a singular race to build the biggest model to a more nuanced approach centered on efficiency, specialization, and new architectural breakthroughs. This period is less of a plateau and more of a pivot, with the potential for significant, nonlinear progress still on the horizon.
Not sure how logical it is to conclude his statement “maybe in the next decade” is remotely equivalent to Sam Altman level hype. That seems like an erroneous stretch on your part born out of emotion. Is it possible that you are overreacting out of cynicism? Is it possible to go too far in the opposite direction of hype toward complete cynicism?
Though the LLM paradigm has its flaws, as he said, science can be self-correcting and there is not a dearth of new ideas. I like Arthur C. Clarke’s statement about how we tend to overestimate technological advancements in the short term but underestimate them in the long run. My personal estimate for an AI system reaching average human level general ability is 50% by 2040 (plus or minus 5 years). I do not believe human cognition represents some sort of cosmic global maximum.
I've been a subscriber since I first found your newsletter & have really appreciated the focus on science and the computer science/math principles behind what these data structures can actually do.
I'll appreciate your continued insights into how LLMs can be a piece of a puzzle and how you'd see organizing data further in ways helpful.
In some ways it seems like we should be able to build reliable systems out of unreliable LLMs, similar to RAID and distributed systems on unreliable commodity hardware.
Likewise...passed the Gary Marcus Substack letter onto my investment advisor, who unsurprisingly have clients begging him to "put us into AI!" at any cost. Gary is a must-read for anybody looking for objective, science- and reality-based critiques and global overview of the SOTA in LLMs today.
You provably need counterfactual, causal inference capabilities, not "glorified regressors" (to quote Prof Judea Pearl) to get "reasoning". No amounts of LLMs will get you anything if the above are not formally addressed (which is freaking hard from a mathematical point of view)
That’s where the smart money should be going right now. Have a few LLMs working on each other, prioritise known good data sets, use them to point where humans could best intervene, and we might actually have something useful!
'AGI—hopefully safe, trustworthy AGI– will eventually come. Maybe in the next decade'
The word maybe is doing some heavy lifting there.
Alternate prediction.
'Artificial General Intelligence' will never, ever, ever happen.
There will be, certainly, breathless exhortations every few months or years of the next milestone attained, or supposed display of autonomous reasoning. The warehouses of energy guzzling server stacks will remain nothing more than electronic parroting devices. Silicon simulacra.
There will be periodic testimonials of consciousness revealed, followed by panel discussions of the need to establish the individual rights of machines.
For those whose (lucrative) professional career has been founded upon the gospel of the user interface, besotted with code, disciples of the wonders of the virtual, more real than real (a version of Plato's forms, I suppose, used without the author's permission, of course), there is too great an emotional investment to recognize that endlessly layering sorting routines at an increased rate will not yield 'Artificial General Intelligence'. When a person has constructed their entire identity on the arrival of the techno rapture, their abiding faith will remain unshaken despite each next failure. Upton Sinclair remains ever salient- 'It's hard to make a man understand when his paycheck depends on him not understanding.'
Here's the thing.
The manifestations of intelligence in living creatures do not involve computation.
Let's repeat that, for the sake of emphasis-
The manifestations of intelligence in living creatures do not involve computation. Because cognition is not computation. (Computation is simply a product of cognition, but then so is flower arrangement.)
No amount of laminated computation will beget emergent properties of intelligence, let alone awareness, let alone self-awareness.
Absent sentience, no intelligence in absentia.
Now, it is worth noting that computers are repositories of human intelligence. Every operation they perform has been constructed (over decades) by hundreds of thousands of individuals, laboring in hardware engineering, manufacturing, coding, mathematics, materials science, etc.
The output of computers is an agglomeration of human thought, extruded. Not unlike the development of language and mathematics over thousands of years (not coincidentally in the least), such human artefacts prompt further discoveries about our world. Studying the elements of math and language produce new information about our world. Pretty cool. But at no point do math or language become entities that operate independently of the humans that manipulate them.
Every supposed demonstration of 'Artificial General Intelligence' has been, ultimately, an elaborate vaudeville act, put-ons by tech bro hucksters, who because of their own stunted imagination and pathological personalities assume they comprehend human cognition. They don't. But they have sold the gullible on their scam, and stoked the delusions of those who simply want to believe.
I share a lot of Gary’s skepticism, but as someone who works in the medical field I have been impressed with GPT-5’s medical knowledge and its ability to apply that knowledge to various clinical scenarios. I have been fact checking against UpToDate, medical textbooks, and the clinical journals and my verdict is so far so good.
Perhaps it's more grounded to say "GPT-5's ability to access [..]" - or "GPT-5's ability to synthesise [..]" - "[..] medical knowledge and it's ability to apply that knowledge to various clinical scenarios".
I offer that because I'm forever encountering edge cases/ mistakes in the domains I know well that amount to poor - if not terrible - responses.
I worry that if the language we use supports reinforcing a mindset of default acceptance of the outputs of these current LLM-based tools - outputs that are derived in the absence of either an anchoring contextual world-model or one or more alternative "2nd opinion" validations of the output - that will ultimately result in cases of significant harm.
I have no experience as a medical practitioner, however my experimental attempts at self-diagnosis using GPT/LLM-based systems have been mixed at best.
One of the things I think seems to be missing is the wider-contextual medical model that would cause a medical practitioner to ask questions that would eliminate possibilities or tune a diagnosis.
Instead, I get a mixed, broad diagnosis - often skewed in a direction that isn't relevant - that requires me to correct that skewing by providing additional context that a medical practitioner would have inquired about if not checked first-hand *before* offering a diagnosis.
Imo, from limited fucking around with grok to see how stupid it is. If 'trained' properly and given comprehensive specialized knowledge it has potential be a beneficial adjunct to highly skilled professions. Replacing the skilled ....not so soon.
Are you able to expand on 1) the nature of the "proper" training with "specialized knowledge" (how someone would go about that), and 2) the ways in which / an example of how it can be a regularly useful "beneficial adjunct" to "skilled professions" ?
I ask because I'm generally curious about what patterns of use can be practically, economically, and successfully used.
I am a statistician working in healthcare, and I can tell for a fact that LLMs are DANGEROUS. Language mangling will get you nowhere without solid stats. Do not expect any causal/counterfactual inference (like you would obtain in a clinical trial) because LLMs have provably ZERO capability in that sense. Caveat emptor must be the first rule when using LLMs in critical settings. None of the attempts to use LLMs in my Hospital Trust have been fruitful when proper statistical validation is used.
A stock market crash seems on the cards. While I'm not thrilled at this prospect, maybe it is the best thing that can happen to us right now, if it's not too bad.
“Nobody ever wanted to talk science.”
^ This.
In a recent interview, Dario Amodei dismissed proofs that LLM alignment is impossible as “nonsense” and “gobbledygook”, adding that he considers people who have published proofs like this morally and intellectually unserious.
The gall. People like myself have spent the time and hard work publishing work like this in peer reviewed academic journals.
Meanwhile, neither he, nor any other developer—nor anyone on any of their teams—have bothered to engage the proofs, as they continue to post methodologically flawed, non-peer reviewed experimental results on their websites and all across the ArXiv, claiming “breakthroughs” left and right.
They are doing bad science, as you point out, in large part because they don’t have the time or wherewithal to respect or uphold basic, longstanding, proven scientific norms—such as, you know, bothering with peer review or addressing proofs and other critiques that don’t align with their PR and bottom line.
Amodei is a world class grifter on a par with Sam Bankman Fried... consider shorting Anthropic stock.. oh right, no IPO yet....https://www.youtube.com/watch?v=9vQaVIoEjOM
Thank you for sticking to your guns and speaking the truth with genuine expertise. As Churchill said, 'Truth is incontrovertible. Panic may resent it. Ignorance may deride it. Malice may distort it. But there it is.'
That would have been nice to include :)
It's such a great quote!
Good quote but today truth has been shown to be eminently convertible.
Very postmodern
Flipped indeed!
"AGI, or human-level AI, is 'not a super useful term'" is backpeddaling personified.
and retrenching and unabashedly bullshittting
This reminds me of Columbus discovering America and never being able to admit it wasn't India. Eventually, folks had to admit that discovering America had its own, lesser value.
How could AI based on the median human reality that is the web [LLM] be substantially more than mediocre?
I don't know why anyone is surprised by the results being produced. This system injests both the best and worst of humanity; the genius and the BS. When you average them together—since the system can't reason or discern—of course one is going to get middling, mediocre results in the output.
But it doesn't do averages—it just reflects the most widely held position.
But AI is no longer Indians all the way down.... 🤣
Why is it “bullshiting”? It's, surprisingly acute, as I wrote before: in a world of “scale is all you need” AGI makes sense. Because “scale” is more-or-less one parameter, as you increase it you first match human and then surpass it. AGI is when you match human, ASI is when you bypass it.
In a world where “scale is NOT all you need”… AGI is meaningless: because AI achieve different things that human can do at different times “AI that's approximately equal in capabilities to human” simply would never exist, only “AI that's strictly worse than human” and “AI that's strictly better than human”.
Maybe better to say not that this proclamation is “bullshitting”, but more of a heavy emphasis of the term was bullshit: it was based on flawed hypnotises, hypnotises is, now, disproven… term is no longer useful. End of Story.
Reminds me of Trump saying he was being sarcastic when he said he'd end the war in Ukraine within 24 hours of being elected. Wonder how long it will take Altman to get to his version of "I don't think Putin wants to end the war."
Pilpul.
Hardly.
Or very lame CYA
Paraphrasing Feynman: "Nature doesn't care about your stock price!"
Tell that to the hypesters on "The Street".
I don't think we've hit peak AI, but I do think we've hit peak AI hype cycle. Next stop: the valley of disillusionment, featuring all the companies that have burned billions in this tech.
Nice to hear from Gary about the long overdue change of the public perception. If we are talking about science, my team has been for decades pursuing what I would call SyNe AI (symbolic AI with support from statistics-based — “neural” tools) which is where the real explanatory science should be looking instead even of what today is called neurosymbolic (NeSy) AI which is for most of its practitioners is just sprinkling some hand-crafted knowledge over what still is an LLM substrate, which won’t get us very far scientifically. But this comment belongs to a different discussion, which I hope will become more central in AI — what kind (or kinds) of neurosymbolic symbiosis would be scientifically more promising.
Hi - interesting comment. Do you have some references for the approach? Which I take it would mean an architecture that envisioned both as co-equal components from the start vs Sym as a semantic patch - and a clearly scoped methodological regime for the dataset (vs. scrape the web)?
BTW - I wonder how we will know whether/how many fixes to purported LLM flaws are just symbolic AI kluges hidden under the covers?
Comment streams are not the best place for a reasoned discussion. Yes, we have plenty of references. The easiest are the 2021 and 2024 books, both free access from MIT Press: https://direct.mit.edu/books/oa-monograph/5042/Linguistics-for-the-Age-of-AI and https://direct.mit.edu/books/oa-monograph/5833/Agents-in-the-Long-Game-of-AIComputational . There’s plenty of other materials, including the 2004 Ontological Semantics book, also from MIT Press.
Sergei
There is an obvious other way - Semantic AI - where the language is English, and acts as the transfer mechanism for structure from the head of one person to another, or to a machine. Every word (and the elided words) is represented by an object or an operator representing the particular meaning the word has in context (some words have multiple POS and/or 60-80 meanings). At the moment, we have a vocabulary of 45,000 words, 10,000 wordgroups (a group of words that has a figurative meaning - "behind bars", "a bridge too far").
An insuperable problem for what you are doing is that humans are not good at complex problems (the Four Pieces Limit), whereas a machine can keep all the connections in the network (tens of thousands) in sync while the human fiddles about with a particular detail, staying at or below their limit.
Eh, no. How do you describe meaning? And for what uses? Read some of our stuff. Of course, it’s too much to read, and it would be difficult. But you’ll learn a lot if you try. Cheers!
Lol Gary! Bingo, again. In fact, pre GPT3.5 they humble-bragged about them behaving responsibly by not releasing their hyped up token calculator all in one piece because doing so would be irresponsible - because the LLM was so dangerous. LMAO.
PS: https://medium.com/data-science/openais-gpt-2-the-model-the-hype-and-the-controversy-1109f4bfd5e8
“braggadacio alone cannot yield AGI”… the brag was getting super stale
Seems to me LLMs fit somewhere in what I view as the parsing side of an eventual AI solution.
They can identify traffic lights like a champ, but have no idea what a traffic light is.
Yes - LLM/GPT-based systems are certainly masterful at understanding the forms and patterns / construction of output that is generally pleasing and / or valuable to humans (to a greater or lesser extent).
As I've noted for a while, they will likely have a useful part to play going forward in input and output construction within the context of a distributed / heterogeneous AI system of services that include one or more domain-specific world-model engines.
They will book-end the interaction by providing a translation from and back to human-likable form.
That is to say, they mimic the observable surfaces of objects and behaviors/actions, and thereby provide an interface (but not much more)?
I was specifically noting that LLM/GPT-based systems - where they have ingested a sufficient quantity of text and images - appear capable of both a) interpreting human-created / human-interpretable text and images and b) constructing human-recognisable and human-likable text and images. They are capable of inferring from an input using a corpus sufficient "lexical semantics" and "visual semantics" to understand the form and logical meaning of text and images, and likewise .
With those capabilities, they could conceptually form a useful function at the boundary of a distributed AI system, where other parts of that system including domain-"grounded knowledge" and contextual awareness work in collaboration to enable the creation of well-formed, meaningful, trustable responses to inquiries.
| they mimic the observable surfaces of objects and behaviors/actions,
| and thereby provide an interface (but not much more)
If you're referring to an Interface Theory of Perception, then perhaps that's the case, though I'm not implying or suggesting that - I'm not familiar enough with that theory to comment.
I was reading a desription of a test tn which the AI program was able to predict the positions of planets, but had no idea of the gravitationall forces actually determining their motion and pisition. It did not infer a real model. The movie "Catch Me if You Can" about the imposter and check fraudster Frank Abagnale (starred Leonardo di Caprio) then popped into my head. He mimicked the appearance and behavior of an airline pilot, for example, with none of the underlying knowledge and skill. LLMs have been characterized as supercharged auto complete, which is consistent with mimicry. We mammals do learm from mimicry, but only so much. As a small boy, I saw contractors installing a new furnace in a large hole in our kitchen floor. Days later, my Mother stopped me from attacking the.linoleum with a knife, "putting in a furnace".
Those are great reflections - and I agree. In some respects, it's an alternative viewpoint on Emily M. Bender et. al's "stochastic parrot".
I also appreciated the insights from Baldur Bjarnason's "LLMentalist" / "mechanical psychic" reflections.
PS - I hope you won't mind if I add yours to my growing collection of useful AI analogies.
"And AGI—hopefully safe, trustworthy AGI–will eventually come. Maybe in the next decade."
Oh for heaven's sake. Please stop going full Altman on us! Don't blow up your growing reputation as a reliable guide by throwing out random numbers.
that was very maybe :)
Alas "Gary Marcus says 'AGI will ... come ... in the next decade'" is likely to be the takeaway.
While the progress of the last few years may seem to have slowed, it's more accurate to say that the field is maturing and diversifying. The focus is shifting from a singular race to build the biggest model to a more nuanced approach centered on efficiency, specialization, and new architectural breakthroughs. This period is less of a plateau and more of a pivot, with the potential for significant, nonlinear progress still on the horizon.
Not sure how logical it is to conclude his statement “maybe in the next decade” is remotely equivalent to Sam Altman level hype. That seems like an erroneous stretch on your part born out of emotion. Is it possible that you are overreacting out of cynicism? Is it possible to go too far in the opposite direction of hype toward complete cynicism?
Though the LLM paradigm has its flaws, as he said, science can be self-correcting and there is not a dearth of new ideas. I like Arthur C. Clarke’s statement about how we tend to overestimate technological advancements in the short term but underestimate them in the long run. My personal estimate for an AI system reaching average human level general ability is 50% by 2040 (plus or minus 5 years). I do not believe human cognition represents some sort of cosmic global maximum.
I've been a subscriber since I first found your newsletter & have really appreciated the focus on science and the computer science/math principles behind what these data structures can actually do.
I'll appreciate your continued insights into how LLMs can be a piece of a puzzle and how you'd see organizing data further in ways helpful.
In some ways it seems like we should be able to build reliable systems out of unreliable LLMs, similar to RAID and distributed systems on unreliable commodity hardware.
Likewise...passed the Gary Marcus Substack letter onto my investment advisor, who unsurprisingly have clients begging him to "put us into AI!" at any cost. Gary is a must-read for anybody looking for objective, science- and reality-based critiques and global overview of the SOTA in LLMs today.
So, when will he have clients begging him to get them out of AI?
You provably need counterfactual, causal inference capabilities, not "glorified regressors" (to quote Prof Judea Pearl) to get "reasoning". No amounts of LLMs will get you anything if the above are not formally addressed (which is freaking hard from a mathematical point of view)
That’s where the smart money should be going right now. Have a few LLMs working on each other, prioritise known good data sets, use them to point where humans could best intervene, and we might actually have something useful!
'AGI—hopefully safe, trustworthy AGI– will eventually come. Maybe in the next decade'
The word maybe is doing some heavy lifting there.
Alternate prediction.
'Artificial General Intelligence' will never, ever, ever happen.
There will be, certainly, breathless exhortations every few months or years of the next milestone attained, or supposed display of autonomous reasoning. The warehouses of energy guzzling server stacks will remain nothing more than electronic parroting devices. Silicon simulacra.
There will be periodic testimonials of consciousness revealed, followed by panel discussions of the need to establish the individual rights of machines.
For those whose (lucrative) professional career has been founded upon the gospel of the user interface, besotted with code, disciples of the wonders of the virtual, more real than real (a version of Plato's forms, I suppose, used without the author's permission, of course), there is too great an emotional investment to recognize that endlessly layering sorting routines at an increased rate will not yield 'Artificial General Intelligence'. When a person has constructed their entire identity on the arrival of the techno rapture, their abiding faith will remain unshaken despite each next failure. Upton Sinclair remains ever salient- 'It's hard to make a man understand when his paycheck depends on him not understanding.'
Here's the thing.
The manifestations of intelligence in living creatures do not involve computation.
Let's repeat that, for the sake of emphasis-
The manifestations of intelligence in living creatures do not involve computation. Because cognition is not computation. (Computation is simply a product of cognition, but then so is flower arrangement.)
No amount of laminated computation will beget emergent properties of intelligence, let alone awareness, let alone self-awareness.
Absent sentience, no intelligence in absentia.
Now, it is worth noting that computers are repositories of human intelligence. Every operation they perform has been constructed (over decades) by hundreds of thousands of individuals, laboring in hardware engineering, manufacturing, coding, mathematics, materials science, etc.
The output of computers is an agglomeration of human thought, extruded. Not unlike the development of language and mathematics over thousands of years (not coincidentally in the least), such human artefacts prompt further discoveries about our world. Studying the elements of math and language produce new information about our world. Pretty cool. But at no point do math or language become entities that operate independently of the humans that manipulate them.
Every supposed demonstration of 'Artificial General Intelligence' has been, ultimately, an elaborate vaudeville act, put-ons by tech bro hucksters, who because of their own stunted imagination and pathological personalities assume they comprehend human cognition. They don't. But they have sold the gullible on their scam, and stoked the delusions of those who simply want to believe.
It's a farce, but it's a dangerous farce.
I wish more educators felt the same way as you. Too many are selling their souls to the devil to the detriment of kids.
I share a lot of Gary’s skepticism, but as someone who works in the medical field I have been impressed with GPT-5’s medical knowledge and its ability to apply that knowledge to various clinical scenarios. I have been fact checking against UpToDate, medical textbooks, and the clinical journals and my verdict is so far so good.
Perhaps it's more grounded to say "GPT-5's ability to access [..]" - or "GPT-5's ability to synthesise [..]" - "[..] medical knowledge and it's ability to apply that knowledge to various clinical scenarios".
I offer that because I'm forever encountering edge cases/ mistakes in the domains I know well that amount to poor - if not terrible - responses.
I worry that if the language we use supports reinforcing a mindset of default acceptance of the outputs of these current LLM-based tools - outputs that are derived in the absence of either an anchoring contextual world-model or one or more alternative "2nd opinion" validations of the output - that will ultimately result in cases of significant harm.
I have no experience as a medical practitioner, however my experimental attempts at self-diagnosis using GPT/LLM-based systems have been mixed at best.
One of the things I think seems to be missing is the wider-contextual medical model that would cause a medical practitioner to ask questions that would eliminate possibilities or tune a diagnosis.
Instead, I get a mixed, broad diagnosis - often skewed in a direction that isn't relevant - that requires me to correct that skewing by providing additional context that a medical practitioner would have inquired about if not checked first-hand *before* offering a diagnosis.
Imo, from limited fucking around with grok to see how stupid it is. If 'trained' properly and given comprehensive specialized knowledge it has potential be a beneficial adjunct to highly skilled professions. Replacing the skilled ....not so soon.
Are you able to expand on 1) the nature of the "proper" training with "specialized knowledge" (how someone would go about that), and 2) the ways in which / an example of how it can be a regularly useful "beneficial adjunct" to "skilled professions" ?
I ask because I'm generally curious about what patterns of use can be practically, economically, and successfully used.
I am a statistician working in healthcare, and I can tell for a fact that LLMs are DANGEROUS. Language mangling will get you nowhere without solid stats. Do not expect any causal/counterfactual inference (like you would obtain in a clinical trial) because LLMs have provably ZERO capability in that sense. Caveat emptor must be the first rule when using LLMs in critical settings. None of the attempts to use LLMs in my Hospital Trust have been fruitful when proper statistical validation is used.
Lol—good for you!
A stock market crash seems on the cards. While I'm not thrilled at this prospect, maybe it is the best thing that can happen to us right now, if it's not too bad.
"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."
Richard Feynman, closing words of Challenger disaster report appendix.