New adventures in AI hype: “Our language models are so ‘conscious’ we need to give them rights”
The return of nonsense on stilts, three years later
My DM’s this morning were filled with journalists and friends asking me about a story that increasingly dubious New York Times ran yesterday. This one:
I am not going to read Kevin’s column, and I don’t think you need to, either. What he wrote about coding was wildly naive, and he couldn’t be bothered to ask whether the system would even extend to Pac-Man , let alone debugging. His near-religious endorsement of the imminence of AGI kind of speaks for itself, as does his apparent aversion to consulting seriously with experts who might disagree with his panglossian takes. His shtick is write to with awe, and to think uncritically; I have honestly had enough.
§
That said, we have seen this movie before. The last time I wrote it about, in June 2022, I called it nonsense on stilts.
For those who have forgotten the story, an AI safety engineer at Google, Blake Lemoine, felt that an LLM-based system that nobody remembers anymore called Lamda had achieved “sentience”. The eternally-sharp Abeba Birhane nailed it then with a tweet that she could equally repost today, in Roose’s honor:
The essence of my own argument then, back in June 2022, applies as much to today’s LLM’s as those of three years ago:
To be sentient is to be aware of yourself in the world; LaMDA simply isn’t. It’s just an illusion, in the grand history of ELIZA a 1965 piece of software that pretended to be a therapist (managing to fool some humans into thinking it was human), and Eugene Goostman, a wise-cracking 13-year-old-boy impersonating chatbot that won a scaled-down version of the Turing Test. None of the software in either of those systems has survived in modern efforts at “artificial general intelligence”, and I am not sure that LaMDA and its cousins will play any important role in the future of AI, either. What these systems do, no more and no less, is to put together sequences of words, but without any coherent understanding of the world behind them, like foreign language Scrabble players who use English words as point-scoring tools, without any clue about what that mean.
I am not saying that no software ever could connects its digital bits to the world, a la one reading of John Searle’s infamous Chinese Room thought experiment. Turn-by-turn navigations systems, for example, connect their bits to the world just fine.
Software like LaMDA simply doesn’t; it doesn’t even try to connect to the world at large, it just tries to be the best version of autocomplete it can be, by predicting what words best fit a given context. Roger Moore made this point beautifully a couple weeks ago, critique systems like LaMDA that are known as “language models”, and making the point that they don’t understand language in the sense of relating sentences to the world, but just sequences of words to one another.
Search and replace LaMDA with Claude, and it all still applies. I still don’t remotely see an argument that current models are sentient, nor any argument that scaling a model makes it more conscious, even if it can mimic more language from humans discussing consciousness. Claude does what LaMDA does better because it has more data, but I don’t see any really argument that Claude is any more sentient than a web browser.
Eric Byrnjolffson is often more bullish on AI than I am, but his 2022 commentary on the whole LaMDA affair, too, could be reposted today without changing a word:
Sad that The New York Times fell for it.
§
You can look at what Anthropic is doing (evaluating the “welfare” of its models) from the standpoint of the philosophy of consciousness (asking very reasonably questions like what would count as consciousness? and how we would measure it in an animal or a machine?, and so on), but I think it is better to look at what is happening from the perspective of of commerce. Anthropic is a business (which incidentally neglects to respect the rights of artist and writers who work they nick). I suspect the real move here is simply, as it so often is, to hype the product — basically by saying, hey, look at how smart our product is, it’s so smart we need to give it rights.
Just wait ‘til you see our spreadsheets!
Gary Marcus is shaking his head.
Chatterbotty” (an update for the times of Lewis Carroll’s “Jabberwocky”)
‘Twas guiling and the Valley boyz
Did hype and gamble on the web
All flimsy were the ‘standard’ marks,
Olympic maths o’erplayed
“Beware the Chatterbot, my son!
The free’s that bait, the pro’s that catch!
Beware the ‘Open’ word, and shun
Felonious infringements, natch!”
He took his BS sword in hand;
Long time the LLM he sought—
So rested he by the Knowledge tree
And stood awhile in thought.
And, as in deepest thought he stood,
The Chatterbot, AI’s a-flame Came tripping through the webby wood,
And ad-libbed as it came!
One, two! One, two! And through and through
The Occam’s blade went snicker-snack!
He left it dead, and with its head
He went galumphing back.
“And hast thou slain the Chatterbot?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!”
He chortled in his joy.
‘Twas guiling and the Valley boyz
Did hype and gamble on the web
All flimsy were the ‘standard’ marks,
Olympic maths o’erplayed
Jeez, I was involved in this sort of conversation years ago, it is total utter bllx and always was. There may be three reasons:
1. I think you are right Gary "the real move here is simply, as it so often is, to hype the product — basically by saying, hey, look at how smart our product is, it’s so smart we need to give it rights." Just marketing basically.
2. It is also a form of deliberate disinformation and distraction to take our time and attention away from the real problems we have to address. In Grant Ennis's 9 Frames of Disinformation from his excellent book Dark PR, which we adapted for social media, food and cigarettes, it is part of the programme of taking focus away from a problem - this approach has been fine tuned in so many areas. Grant uses food, transport and climate to make the point.
3. Where I was involved it was the philosophers, who always need new things to pontificate and speculate about signifying not much. So this was just another social science gravy train which might get some cash from the EU.
We have concluded that the best thing is not to dignify it with any further airtime and let it stand alone for everyone to see clearly the total nonsense it is.