You can't have it both ways. You can't say, oh wow, look at our superintelligent machines, way beyond a sackful of PhDs, you can't believe just how clever they are. And then say -- oh, they make the most horrendous mistakes, don't you know, they just make shit up, they do it all the time, it's up to you to check and verify. Don't blame us, it's *your* fault, the user, for using our product and believing what it said.
For me, this is a defining moment. I keep saying, when are they going to call these LLMs out as deceptive technology, as an outright fraud. You can't have it both ways. You can't create a universal encyclopedia that you claim can answer all questions and then say, oh, it makes up stuff all the time.
This has all been deception on a grand scale and it has to end, pronto. This is exactly the way to do it -- you simply make the AI companies responsible for their products. That's all. We have been bewitched by robots and this is how we break the spell. What's so hard about this? Why do AI companies get a pass on being responsible for the dangerous malfunctioning of their products when operated perfectly normally?
I have a screenshot from May 2 2026 of Google AI Overview telling me that 2027 is not next year, it's the year after next. Google AI Overview cannot add 6 + 1. If you bought a calculator and it told you very confidently that 6 + 1 = 6, you would take it back to the shop and get your money back.
Fred Malherbe: In the age of specialization, we all often put our trust in the professional aura and (hopefully) that they are about guarding of their own good name.
It looks to me like Google is messing around with whatever basic trust they have built up over time and probably now take for granted.
They might not know it, but in business, and even though CEOs may not personally give a hoot about name or much else, and though tacit, a good name is already monetized, and its flow cycle occurs in the institutions of a CULTURE. Once the horse is out of that barn, especially with social media today, it's gone.
There's almost nothing in life more fragile than a reputation. A company can spend decades building up its aura and have it completely destroyed in one incident. This court ruling should reverberate across the industry. OMG, we're being held accountable for our products.
A shared remembrance: while teaching an ethics class in a small college in Virginia, I taught from Aristotle's "Nicomachean Ethics"--there was always a great discussion flowing from the section where he treats the importance of one's reputation. I always loved teaching from that text.
Okay, now I have a new book to look up and read, clearly. This is why people love moral philosophy professors, you know. 😉
But more seriously, this is great comment thread atop an incisive article. I sometimes think that we should use the Aladdin's Genie answer to Citizens United. Sure, corporations are people! I can believe that, in much the same way as a colony of honeybees is a singular superorganism. So let's give them ALL the things that come along with being a person, including being able to get thrown in jail, deported from a country, forced to perform mandatory community service...
Danielle Church: . . . some of my most cherished memories . . . to be sure.
But your examples are only those we use when we claim to be civilized. . . . the problem of course, and to use your metaphor, is that they are messing with the hive; and when the hive catches fire, all bees burn and die. (
Also, if you buy the book, it's a translation--get a good copy--mine is from Martin Ostwald, but there are some good but also some real turkeys out there.)
Danielle Church: Also, if you buy the book, it's a translation--get a good copy--mine is from Martin Ostwald, but there are some good but also some real turkeys out there.
Google is being held responsible for the misinformation they shove into our faces. Good. And if this news crashes the share prices, then I have only one thing to say to salty investors: HA
Not sure if that is going to be the outcome. Assuming the logic survives a possible appeal, it may create an expensive compliance framework only incumbent (US) players can afford to follow, inadvertently making it much harder for smaller (absent?) European companies from entering the market in any meaningful way. The net result: more European dependence on US companies.
This is a very important case. I hope. it is upheld, from what I understand they have stronger defamation protections and perhaps one positive outcome of a breakdown of US/EU relations will be them holding US firms to account.
This has the possibility of breaking common carrier-type protections for them, and being much like the Tobacco lawsuits by the the US States. I have always been disappointed by the failure of governments to hold Tesla to account for their failures, perhaps we will see a new dawn of oversight out of this mess.
In the grand scheme of things, this is a very minor error (hallucination), but it cracked me up nonetheless because it involved basic knowledge, not complex music theory. The other day I was transposing a song for our piano player I play in C relative to capo 3 on guitar (a C chord with a capo on the 3rd fret is an Eb). Curious if the Google AI summary would be correct, I asked what the corresponding chord would be for playing without a capo on 3. The summary declared a C shape with capo 3 is a D chord without a capo.
I am not sure I can be confident whether the current Frankensteinian conglomerations of LMMs, stable diffusion and deterministic logic really is not enough. It's probably grossly inefficient, but perhaps a grossly inefficient mechanism with enough energy wasted can still work.
Sure, they may have had to spend 10 million dollars of electricity, sift through countless pages of nonsense hallucinations, provide some hints in the right direction, and polish up the output, but they did solve the unit distance problem. Facebook's support chat may be easily convinced to hand out account information, and might also be unsustainably expensive, but it still can do a wide number of things. The largest models may be overhyped, and again, they may be using ridiculous amounts of electricity behind the scenes, but being able to implement working hacks for an eighth of software vulnerabilities or whatever it was is no joke.
But I am certainly unsure whether even that can get one "all the way," as it were, so I will definitely concede that point.
But I think the question of whether AI should be huge is really one that is going mostly unaddressed. A little while back, you shared a quotation from an essay by Émile Torres, where they suggested that many young AI researchers want the extinction of human beings, but the actual essay was even bleaker and more depressing, if possible.
They argued that a majority of company leaders, not just young researchers, want the extinction of biological humans, that such an outcome is all but guaranteed with the current philosophies of the people in charge, with the only question being whether AIs take over and kill everyone or they instead remain controlled and enable human-hating CEOs to kill everyone, and that even if people wanting coexistence were in charge, a more likely (they claim much more likely) result would be developing models that simply would lead to a slower extinction of humans, if we are lucky! I think some of this is speculative and can be argued against, but most of it is based on clear facts.
And as I said, it is an extremely bleak claim if it is taken at least halfway seriously, and I do, not least because the only people who showed up in the comment section to "debunk" elements of it seemed to instead be advocating for exactly what it warned about, but framed in less stark terms. A lot less “how can they make these ridiculous claims that AI advocates want to kill all humans and are likely to do so” and more “people should just accept that human beings all being replaced by robots is not a bad thing.”
With this kind of thing in mind, should we not be asking whether AI should be developed at all, at least in the world as it is today?
At the hearing, Google argued users generally knew AI generated information should not be blindly trusted. The court said the AI overviews are Google's own content and rejected the argument users were responsible for fact-checking the results themselves.
Although I think that Google should be held to account for their “facts”, I also think that a person has to be foolish NOT to check the “facts” that these bots spit out.
They have been demonstrated not to be reliable. If the answer is at all important, even a 90% accuracy rate requires verification every time because (without checking) there is no way of knowing if one has received something in the 10% incorrect category.
That severely limits the uses of the bots to things that don’t really matter because if one has to check and verify every answer, there is no point in even using the bots.
The German court made the distinction between Google's search links and the AI Summary.
Search links point to the actual information sources whose veracity Google is not responsible.
The AI Summary, which is an amalgam of information sources including from the Gemini LLM is created by Google and as such it is responsible for its veracity.
You can't have it both ways. You can't say, oh wow, look at our superintelligent machines, way beyond a sackful of PhDs, you can't believe just how clever they are. And then say -- oh, they make the most horrendous mistakes, don't you know, they just make shit up, they do it all the time, it's up to you to check and verify. Don't blame us, it's *your* fault, the user, for using our product and believing what it said.
For me, this is a defining moment. I keep saying, when are they going to call these LLMs out as deceptive technology, as an outright fraud. You can't have it both ways. You can't create a universal encyclopedia that you claim can answer all questions and then say, oh, it makes up stuff all the time.
This has all been deception on a grand scale and it has to end, pronto. This is exactly the way to do it -- you simply make the AI companies responsible for their products. That's all. We have been bewitched by robots and this is how we break the spell. What's so hard about this? Why do AI companies get a pass on being responsible for the dangerous malfunctioning of their products when operated perfectly normally?
I have a screenshot from May 2 2026 of Google AI Overview telling me that 2027 is not next year, it's the year after next. Google AI Overview cannot add 6 + 1. If you bought a calculator and it told you very confidently that 6 + 1 = 6, you would take it back to the shop and get your money back.
Fred Malherbe: In the age of specialization, we all often put our trust in the professional aura and (hopefully) that they are about guarding of their own good name.
It looks to me like Google is messing around with whatever basic trust they have built up over time and probably now take for granted.
They might not know it, but in business, and even though CEOs may not personally give a hoot about name or much else, and though tacit, a good name is already monetized, and its flow cycle occurs in the institutions of a CULTURE. Once the horse is out of that barn, especially with social media today, it's gone.
There's almost nothing in life more fragile than a reputation. A company can spend decades building up its aura and have it completely destroyed in one incident. This court ruling should reverberate across the industry. OMG, we're being held accountable for our products.
Fred Malherbe: OMG indeed.
A shared remembrance: while teaching an ethics class in a small college in Virginia, I taught from Aristotle's "Nicomachean Ethics"--there was always a great discussion flowing from the section where he treats the importance of one's reputation. I always loved teaching from that text.
Okay, now I have a new book to look up and read, clearly. This is why people love moral philosophy professors, you know. 😉
But more seriously, this is great comment thread atop an incisive article. I sometimes think that we should use the Aladdin's Genie answer to Citizens United. Sure, corporations are people! I can believe that, in much the same way as a colony of honeybees is a singular superorganism. So let's give them ALL the things that come along with being a person, including being able to get thrown in jail, deported from a country, forced to perform mandatory community service...
Danielle Church: . . . some of my most cherished memories . . . to be sure.
But your examples are only those we use when we claim to be civilized. . . . the problem of course, and to use your metaphor, is that they are messing with the hive; and when the hive catches fire, all bees burn and die. (
Also, if you buy the book, it's a translation--get a good copy--mine is from Martin Ostwald, but there are some good but also some real turkeys out there.)
Danielle Church: Also, if you buy the book, it's a translation--get a good copy--mine is from Martin Ostwald, but there are some good but also some real turkeys out there.
“Having it bots’ ways”
My bots are PhD
Material, can’t you see?
But DON’T you blame my bots
That MIS-count seven dots
Google is being held responsible for the misinformation they shove into our faces. Good. And if this news crashes the share prices, then I have only one thing to say to salty investors: HA
Not sure if that is going to be the outcome. Assuming the logic survives a possible appeal, it may create an expensive compliance framework only incumbent (US) players can afford to follow, inadvertently making it much harder for smaller (absent?) European companies from entering the market in any meaningful way. The net result: more European dependence on US companies.
You're not reading the room very well
Does this only cover hallucinations by their bots?
Or does it also cover hallucinations by company officials?
Dotcom bubble was a scam, though.
Investors foolishly bought into breathless, irrational hysterics about how they had to throw money at dot com stocks.
So for this guy to say "this scam is 50x bigger than the previous scam!" is not the compliment that he thinks it is.
“The Botcon Bubble”
Botcon bubble
Bigger still
Than dotcom trouble
Run of mill
Yeah 50x bigger than Dotcom and 50x bigger bust
I wish I knew who Masa was. Of course I could just look him up, but we know how trustworthy Google is, so I'd rather hear it from Gary than Google.
In case this was a real question ;-) he is Masayoshi Son, Softbank CEO. Wikipedia seems still reliable :-)
The Son of Bot, no doubt.
The Father, the Son and the Wholey Boast
In Shintoism, I don't think there is a trinity :-)
I thought we were talking about Botism.
Isn’t he a Botist?
Albeit most definitely not a modest Botist
I don't know, but maybe he's channeling his forebears by flying a (financial) Ohka? (https://en.wikipedia.org/wiki/Yokosuka_MXY-7_Ohka)
I wondered for a long time why he's so hell bent on wasting his company's money and the money of (potentially dangerous) lenders.
We will know when the Singularity has arrived because the bots will take over Wikipedia, plant a flag in it and claim it for Bot and King
He's the genius who said WeWork was going to be huge and then bailed out its founder because he was smitten with him
This is a very important case. I hope. it is upheld, from what I understand they have stronger defamation protections and perhaps one positive outcome of a breakdown of US/EU relations will be them holding US firms to account.
This has the possibility of breaking common carrier-type protections for them, and being much like the Tobacco lawsuits by the the US States. I have always been disappointed by the failure of governments to hold Tesla to account for their failures, perhaps we will see a new dawn of oversight out of this mess.
In the grand scheme of things, this is a very minor error (hallucination), but it cracked me up nonetheless because it involved basic knowledge, not complex music theory. The other day I was transposing a song for our piano player I play in C relative to capo 3 on guitar (a C chord with a capo on the 3rd fret is an Eb). Curious if the Google AI summary would be correct, I asked what the corresponding chord would be for playing without a capo on 3. The summary declared a C shape with capo 3 is a D chord without a capo.
Man, I just wrote up a bunch of these cases in yesterdays article. Never fails, the big ones hit right after you publish. Thanks for calling it out.
(proofreading note, there is a space at the beginning of the article title)
None of the stock of these AI companies is worth the paper it is printed on. So much circular financing, it makes me dizzy
Definitely not worth the electrons in the GPUs used to do the calculations.
100% behind this decision.
Should AI be huge, though?
I am not sure I can be confident whether the current Frankensteinian conglomerations of LMMs, stable diffusion and deterministic logic really is not enough. It's probably grossly inefficient, but perhaps a grossly inefficient mechanism with enough energy wasted can still work.
Sure, they may have had to spend 10 million dollars of electricity, sift through countless pages of nonsense hallucinations, provide some hints in the right direction, and polish up the output, but they did solve the unit distance problem. Facebook's support chat may be easily convinced to hand out account information, and might also be unsustainably expensive, but it still can do a wide number of things. The largest models may be overhyped, and again, they may be using ridiculous amounts of electricity behind the scenes, but being able to implement working hacks for an eighth of software vulnerabilities or whatever it was is no joke.
But I am certainly unsure whether even that can get one "all the way," as it were, so I will definitely concede that point.
But I think the question of whether AI should be huge is really one that is going mostly unaddressed. A little while back, you shared a quotation from an essay by Émile Torres, where they suggested that many young AI researchers want the extinction of human beings, but the actual essay was even bleaker and more depressing, if possible.
They argued that a majority of company leaders, not just young researchers, want the extinction of biological humans, that such an outcome is all but guaranteed with the current philosophies of the people in charge, with the only question being whether AIs take over and kill everyone or they instead remain controlled and enable human-hating CEOs to kill everyone, and that even if people wanting coexistence were in charge, a more likely (they claim much more likely) result would be developing models that simply would lead to a slower extinction of humans, if we are lucky! I think some of this is speculative and can be argued against, but most of it is based on clear facts.
And as I said, it is an extremely bleak claim if it is taken at least halfway seriously, and I do, not least because the only people who showed up in the comment section to "debunk" elements of it seemed to instead be advocating for exactly what it warned about, but framed in less stark terms. A lot less “how can they make these ridiculous claims that AI advocates want to kill all humans and are likely to do so” and more “people should just accept that human beings all being replaced by robots is not a bad thing.”
With this kind of thing in mind, should we not be asking whether AI should be developed at all, at least in the world as it is today?
At the hearing, Google argued users generally knew AI generated information should not be blindly trusted. The court said the AI overviews are Google's own content and rejected the argument users were responsible for fact-checking the results themselves.
Although I think that Google should be held to account for their “facts”, I also think that a person has to be foolish NOT to check the “facts” that these bots spit out.
They have been demonstrated not to be reliable. If the answer is at all important, even a 90% accuracy rate requires verification every time because (without checking) there is no way of knowing if one has received something in the 10% incorrect category.
That severely limits the uses of the bots to things that don’t really matter because if one has to check and verify every answer, there is no point in even using the bots.
The German court made the distinction between Google's search links and the AI Summary.
Search links point to the actual information sources whose veracity Google is not responsible.
The AI Summary, which is an amalgam of information sources including from the Gemini LLM is created by Google and as such it is responsible for its veracity.
I was referring to the AI generated material (AI summary)
Finally!!! Thanks for this good news; it was mos def not a bother.
Good insight 😃. Can i translate this article into Spanish with links to you and a description of your newsletter?