The Royal Society publishes presentation on their YouTube channel (https://www.youtube.com/@royalsociety) so hopefully we'll see this one appear there sooner or later.
My pet peeve when it comes to the Turing Test are all the useless claims that modern LLMs pass it. I would love to see that addressed at the meeting. IMHO, what we need is a new formulation of the Turing Test where the inner workings of the AI matter. Of course, this would contradict Turing's idea that if the human questioner can't tell the difference between AI and human, then there is no difference and the AI passes the test. This didn't anticipate LLM "cheating" so we need to add some qualifications that both the AI and the human questioner must pass in order for the test to be valid.
The people that have attempted to execute the Turing Test on AI have not followed the guidelines well. It’s supposed to be at least 15 minutes but most cap it at that. Most people can’t tell if someone is lying either so are poor candidates to question AI. I can guarantee I could expose AI quickly knowing it’s flaws.
The Turing Test is not a test of intelligence. It fails to note what any anthropologist could tell you, which is that most human interchanges are ritualised exchanges of attention, which are nothing to do with intelligence.
Therefore, the Test just shows how little Turing and the AI community know about social psychology.
The AI community is heavily populated with autistic people, who struggle to understand normal human interactions, and they tend to ignore widely known knowledge from other academic fields.
This also explains why LLMs are millions of times more energy-hungry than mammalian brains. They have chosen to ignore just about everything that has been learned about how neurons in mammalian brains really work since 1943.
IMO the Turing Test can be improved via *other* tests of intelligence - which are done by (for example) forensic clinical psychologists in very interesting circumstances, even administered to the non-verbal. (I'm just reporting - my sister's profession, not mine.)
Any test designed for human subjects will likely suffer from biases that make sense for humans but are unreasonable for AI. LLMs now pass all kinds of tests meant for humans but they do so by "memorizing" the entire internet, something the test creators assume a human subject couldn't do. They also assume that if the test subject can correctly answer a question, then the test taker knows the subject, not just the word-order statistics that make up an acceptable answer.
Turing's imitation game as an operational definition of thinking was based on the assumption that the machine would use *internal logic* to drive the teletype and present the behavior to be judged, not merely do statistical pattern matching against a vast collection of human texts--something Turing could not have conceived of in 1950, and he surely would have rejected *that* sort of system as demonstrating machine thinking, just as he would reject the Mechanical Turk as showing that the machine can play chess--it was the man inside who played chess, and it's the vast number of people who furnished the texts comprising the training data who can think. The Turing Test (called the Imitation Game by Turing) was not some ploy to gain market share, it was to provide a concrete basis for judging future technology as having produced a machine that thinks. If things that clearly *don't* think are passing the test then the test needs strengthening to reject those end-arounds.
To be fair, many of the generative chatbots like ChatGPT / Gemini / Claude etc. are now capable of passing a "short" version of the Turing Test. We are easily fooled by anthropomorphism and the ELIZA effect. They can even go unnoticed by experienced judges.
As a seasoned NLP practitioner with a recent PhD (2020) in deep learning, my observations, particularly regarding the stunning progress in machine translation, once considered the holy grail of AI, lead me to hypothesize that many language processing tasks require little reasoning. They are primarily System 1 tasks, fast and intuitive, as described by the late Daniel Kahneman, who sadly passed away recently.
I think we should not see the Turing Test as a "intelligence test," but rather an epistemological thought experiment. Turing Test, aka thought experiment, is too much based on NLP to be a broader test for AGI.
Establishing an efficient and reliable protocol to determine whether an AI system is "truly intelligent", "AGI level" or "conscious" is a challenging problem—perhaps even an ill-posed or intractable one without any satisfactory solution. By the way, I'm a big fan or François Chollet's ARC tests.
Anyway, if intelligence becomes artificial, let's hope it frees us from our natural stupidity, particularly our actual inaction in the face of the environmental crisis.
“A few tickets have just been released to the public, details here.” The live event has been sold out since the day this was posted; but EventBright does offer Free livestream tickets. Sadly, there are no links or instructions how to join said livestream.
Anybody have any suggestions (since it starts soon) ??
The event was amazing (Gary, but also all of the other speakers!); it's worth taking the time to watch it; available on YouTube https://www.youtube.com/live/GmnBTCKocZI
I am very disappointed that Eventbrite failed to send me a link to join the livestream (as had been promised). And the QR code on the ticket, did not resolve to a stream link.
I am stunned that even educated people claim that AIs have passed the Turing Test. Consider Ethan Mollick's book, "Co-Intelligence," among many other notable publications! I am constantly attacked by consultants and so-called AI experts on panels whenever I mention that no machine has ever passed the test, even if one can criticize it. I started in AI in 1989, and I still hope that one day investments will shift towards alternative architectures rather than scaling vulnerable transformers/ old approaches around ANNs.
I'm looking forward to seeing your presentation on YouTube.
We need to increase AI literacy among the general public. People can't ask good questions without understanding the basics of AI. Right now, AI gets elevated into the state of religion, with big churches and their priests, an army of worshippers, and public cancellations of heretics who doubt outrageous nonsense around AGI and who dare to speak up. As a result, value is destroyed, and only 95 percent of AI pilots succeed, while only a few benefit from what are very useful technologies.
The Royal Society publishes presentation on their YouTube channel (https://www.youtube.com/@royalsociety) so hopefully we'll see this one appear there sooner or later.
My pet peeve when it comes to the Turing Test are all the useless claims that modern LLMs pass it. I would love to see that addressed at the meeting. IMHO, what we need is a new formulation of the Turing Test where the inner workings of the AI matter. Of course, this would contradict Turing's idea that if the human questioner can't tell the difference between AI and human, then there is no difference and the AI passes the test. This didn't anticipate LLM "cheating" so we need to add some qualifications that both the AI and the human questioner must pass in order for the test to be valid.
for sure i will address these issues
The people that have attempted to execute the Turing Test on AI have not followed the guidelines well. It’s supposed to be at least 15 minutes but most cap it at that. Most people can’t tell if someone is lying either so are poor candidates to question AI. I can guarantee I could expose AI quickly knowing it’s flaws.
No LLM has passed the Turing test. Don’t believe those wankers.
The Turing Test is not a test of intelligence. It fails to note what any anthropologist could tell you, which is that most human interchanges are ritualised exchanges of attention, which are nothing to do with intelligence.
Therefore, the Test just shows how little Turing and the AI community know about social psychology.
The AI community is heavily populated with autistic people, who struggle to understand normal human interactions, and they tend to ignore widely known knowledge from other academic fields.
This also explains why LLMs are millions of times more energy-hungry than mammalian brains. They have chosen to ignore just about everything that has been learned about how neurons in mammalian brains really work since 1943.
Music to my ears.
I would have come to the event, but I won’t be in England that day, where I normally live 😞
This is the same conundrum facing teachers: did that student pass the Turing test?
There’s a variant on this conundrum: did the student pass the LLM test? In other words, did they not bother doing any intelligent search at all?
IMO the Turing Test can be improved via *other* tests of intelligence - which are done by (for example) forensic clinical psychologists in very interesting circumstances, even administered to the non-verbal. (I'm just reporting - my sister's profession, not mine.)
Any test designed for human subjects will likely suffer from biases that make sense for humans but are unreasonable for AI. LLMs now pass all kinds of tests meant for humans but they do so by "memorizing" the entire internet, something the test creators assume a human subject couldn't do. They also assume that if the test subject can correctly answer a question, then the test taker knows the subject, not just the word-order statistics that make up an acceptable answer.
The turing test never had any value to begin with. I think Turing was being sarcastic. Has everyone forgotten he was British?
^^ Gary, you can quote me in your speech if you give me credit.
Ooh congrats, Gary!
For God's sake, don't drive on the right (wrong) side of the road!
I will be there, albeit virtually from California.
Me too--from California.
Booked! Looking forward to seeing you there Gary.
In-person ticket sold out 😭😭
Yours truly! And congrats!
Really sorry they scheduled it on Yom Kippur. Insensitive.
hopefully will be recorded
No that's not "insensitive", you self-centered git.
The Turing Test turns out not to test if a machine can think but if people can be made to believe that a machine can think.
Turing's imitation game as an operational definition of thinking was based on the assumption that the machine would use *internal logic* to drive the teletype and present the behavior to be judged, not merely do statistical pattern matching against a vast collection of human texts--something Turing could not have conceived of in 1950, and he surely would have rejected *that* sort of system as demonstrating machine thinking, just as he would reject the Mechanical Turk as showing that the machine can play chess--it was the man inside who played chess, and it's the vast number of people who furnished the texts comprising the training data who can think. The Turing Test (called the Imitation Game by Turing) was not some ploy to gain market share, it was to provide a concrete basis for judging future technology as having produced a machine that thinks. If things that clearly *don't* think are passing the test then the test needs strengthening to reject those end-arounds.
To be fair, many of the generative chatbots like ChatGPT / Gemini / Claude etc. are now capable of passing a "short" version of the Turing Test. We are easily fooled by anthropomorphism and the ELIZA effect. They can even go unnoticed by experienced judges.
As a seasoned NLP practitioner with a recent PhD (2020) in deep learning, my observations, particularly regarding the stunning progress in machine translation, once considered the holy grail of AI, lead me to hypothesize that many language processing tasks require little reasoning. They are primarily System 1 tasks, fast and intuitive, as described by the late Daniel Kahneman, who sadly passed away recently.
I think we should not see the Turing Test as a "intelligence test," but rather an epistemological thought experiment. Turing Test, aka thought experiment, is too much based on NLP to be a broader test for AGI.
Establishing an efficient and reliable protocol to determine whether an AI system is "truly intelligent", "AGI level" or "conscious" is a challenging problem—perhaps even an ill-posed or intractable one without any satisfactory solution. By the way, I'm a big fan or François Chollet's ARC tests.
Anyway, if intelligence becomes artificial, let's hope it frees us from our natural stupidity, particularly our actual inaction in the face of the environmental crisis.
“A few tickets have just been released to the public, details here.” The live event has been sold out since the day this was posted; but EventBright does offer Free livestream tickets. Sadly, there are no links or instructions how to join said livestream.
Anybody have any suggestions (since it starts soon) ??
Links arrived early (Eastern Time) this morning; now there’s a queue of about 72 waiting …
Gary, say hi to Alan for me. This should be a great event!
Will be glued to the livestream! In person tickets all gone already. Will you be at any other events whilst in the UK?
The event was amazing (Gary, but also all of the other speakers!); it's worth taking the time to watch it; available on YouTube https://www.youtube.com/live/GmnBTCKocZI
I am very disappointed that Eventbrite failed to send me a link to join the livestream (as had been promised). And the QR code on the ticket, did not resolve to a stream link.
It's not at the Royal Society's YouTube page: https://www.youtube.com/@royalsociety/streams (or the Turing Institute, etc)
I am stunned that even educated people claim that AIs have passed the Turing Test. Consider Ethan Mollick's book, "Co-Intelligence," among many other notable publications! I am constantly attacked by consultants and so-called AI experts on panels whenever I mention that no machine has ever passed the test, even if one can criticize it. I started in AI in 1989, and I still hope that one day investments will shift towards alternative architectures rather than scaling vulnerable transformers/ old approaches around ANNs.
I'm looking forward to seeing your presentation on YouTube.
We need to increase AI literacy among the general public. People can't ask good questions without understanding the basics of AI. Right now, AI gets elevated into the state of religion, with big churches and their priests, an army of worshippers, and public cancellations of heretics who doubt outrageous nonsense around AGI and who dare to speak up. As a result, value is destroyed, and only 95 percent of AI pilots succeed, while only a few benefit from what are very useful technologies.