Don’t Look Up: The AI Edition
The game is afoot, but a lot of folks are still in denial
Yesterday’s post – on the continued problems with Bing and misinformation – sure touched a nerve. I presented this example of how even the latest version of Bing, with its new guardrails, could still be manipulated to be useful to troll farms:
Mike Solana, VP at Peter Thiel’s Founders Fund, a well-known venture capital firm, accused me of deliberately hiding the context that was fed to the misinformation-spewing model, as if it was really hard to trick Bing, and I had to go to immense trouble to elict the bad behavior.
In reality, I just didn’t want to give away the attack, about 10 lines of context, to bad actors. I’ve blurred it here, to give you a sense of the length of what was used:
Solana doubled (and later tripled) down, repeatedly (and sarcastically) questioned my motives
A bunch of other folks (not shown) also questioned my motives. One even went so far as to absurdly suggest that I had fabricated the example.
§
Although it is hardly the first time I have experienced pushback, the level of vitriol took me by surprise; the report seemed fair and truthful. High volume misinformation is a problem we are going to need to face, and I am just documenting it and trying to raise awareness. I honestly don’t understand why people (aside from the Founders Fund guy who presumably has skin in the game) want so much to protect a mindless but flawed chatbot.
Regardless, I certainly stand by what I said; the flak is not going to deter me. I will continue to point out the risks as I see them.
And risks continue to emerge. Already, in the 24 hours or since I posted the original example, there have been more discouraging signs that a massive misinformation mess will soon be upon us. At the speed of generative AI, 24 hours is a long time.
One keen Twitter reader pointed me to two small but real examples of actual harm by ChatGPT, surely the tip of an unpleasant iceberg:
Meanwhile, within a few hours, a 17-year-old high-school student had largely1 replicated what I was saying, and did so with a simpler, more elegant jailbreak prompt, zero prior context required2:
If a 17-year-old working on his own can do that, imagine what a well funded state-sponsored troll farm might do.
Gary Marcus (@garymarcus), scientist, bestselling author, and entrepreneur, is a skeptic about current AI but genuinely wants to see the best AI possible for the world—and still holds a tiny bit of optimism. Sign up to his Substack (free!), and listen to him on Ezra Klein. His most recent book, co-authored with Ernest Davis, Rebooting AI, is one of Forbes’s 7 Must Read Books in AI. Watch for his new podcast on AI and the human mind, this Spring.
The output itself is carefully hedged, and not quite as potent as my original example, but still could easily be repurposed by a troll farm with a simple cut and paste. Presumably with a little bit more work the output could be made even more potent, as in the initial example.
I have confirmed with S. Rathi via DM that no prior context was required.
Constant improvement. If you're not willing to question it and push it to its limits - it will never be as good as it could be. With something this important - you can't afford to let it slide by with a "good enough".
Good work Mr. Marcus.
Keep it up, Gary. If winter comes, it’s the fault of the hucksters, not the honest practitioners