Two frightening things before breakfast
Two frightening things before breakfast
We are living in the Wild AI West. Get used to it.
I woke up early this morning, and I can’t go back to sleep. My mistake was to check my email. Two things that I have been worrying about for months suddenly feel much more imminent than I originally thought.
§
My first worry, which I have written about frequently, was that Large Language Models might be co-opted by bad actors to produce misinformation at large scale, using custom-trained models. Back in December, I told Ezra Klein that I thought that this could happen, and that a troll farm might be able to make custom-version of GPT-3 for under half a million dollars.
§
I was wrong. By three orders of magnitude.
What I learned this morning is that retraining GPT-3 in some new political direction isn’t a half-million dollar proposition.
It’s a half-a-thousand dollar proposition.
David Rozado, who had recently established the left-leaning tendencies
of ChatGPT
was curious would it take to get GPT-3 to lean the other way. Turns out the answer is not much. He summarizes in this Twitter thread, which links to his Substack:
His results speak for themselves
But what blew my mind was the cost.
The implications for our political system, and future elections, are profound.
§
Oh, and for good measure the tooling for spreading automatically generated misinformation is getting more efficient, too:
§
A second ongoing worry that I have had has been that chatbots were going to get someone killed.
I may have first mentioned this concern out loud in August—a long, long time ago, in the era before ChatGPT-3, before Galactica, before the new Bing, and before Bard—when an editor at WIRED wrote to me, asking for a prediction for 2023.
He greenlit the piece; I wrote it up. WIRED ran in December 29, under the title The Dark Risk of Large Language Models. Here’s the first paragraph:
Even yesterday, when I mentioned this scenario in another interview, it still felt somewhat speculative. That was then.
This morning, after I read the Rozado study and already couldn’t get back to sleep, a friend sent me Kevin Roose’s new column, saying “there is something quite disturbing about it….
And here’s an excerpt, Roose in bold, the rest generated by Bing.
It’s not hard to see how someone unsophisticated might be totally taken in, by this prevaricating but potentially emotionally damaging hunk of silicon, potentially to their peril, should the bot prove to be fickle. We should be worried; not in the “some day” sense, but right now.
§
As a final word, I am on a business trip and wrote yesterday’s piece (Bing’s Rise and Fall) and this one in incredible haste (sorry for the typos!). There was a broken image link, and I went so fast I neglected to mention the personally salient fact that Bing had lied about me, claiming that I believe that “Bing’s AI was more accurate and reliable than Bard”, when I never said any such thing.
What I actually said was quite different, almost scandalously so: “Since neither company has yet subjected their products to full scientific review, it’s impossible to say which is more trustworthy; it might well turn out that Google’s new product is actually more reliable.”
§
Moments later, after I posted yesterday, I discovered that I was outdone by a better, more in-depth blog on the same point of Bing’s troubled recent behavior, with more examples and deeper analysis. I quote from its conclusion here:
In support of LessWrong’s sharp observation that “better” models might be in some way less good for humanity, (“agentic failures are harder to remove in moer capable/larger models”) I present this mindblowing example from Ethan Mollick from the new Bing, that is simultaneously more fluent but less true:
It’s hard to fathom what bad actors might do with such tools.
§
There are almost no regulations here; we are living in the Wild AI West. Microsoft can (at least to a first approximation) do whatever the hell it wants with Bing.
I am not sure that is what we want.
Gary Marcus (@garymarcus), scientist, bestselling author, and entrepreneur, is a skeptic about current AI but genuinely wants to see the best AI possible for the world—and still holds a tiny bit of optimism. Sign up to his Substack (free!), and listen to him on Ezra Klein. His most recent book, co-authored with Ernest Davis, Rebooting AI, is one of Forbes’s 7 Must Read Books in AI.
Strictly speaking, as I discussed in Inside the Heart of ChatGPT’s Darkness, ChatGPT doesn’t literally lean left or right per se, because it doesn’t have political ideas. What political things it espouses are a stochastic function of the corpus it is parroting.
Omg Gary. Thank you for continuing to highlight the dark side of all this. With such a low-cost barrier to entry, 'scary' doesn't start to describe the potential for misuse.
Hi Gary, your concerns here are entirely valid and the argument you present on these dangers is airtight.
Perhaps now, more than ever, we should lend our minds to the idea that Computational Psychology and Psychiatry are both emergent fields of science and medicine, that can, overall, be the most effective countering forces in the coming future, in the face of this kind of pervasive and destabilizing influence.