There are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.
- Donald Rumsfeld
Large language models can be harnessed for good; let there be no doubt about that. But they are almost certain to cause harm as well, and we need to prepare for that, too. And what really scares me is that I don’t think we yet know the half of what that entails.
In previous posts, I had already pointed to at least five concerns:.
State actors and extremist groups are likely to use large language models to deliberately mass produce authoritative-sounding misinformation with fake references and data at unprecedented scale, attempting to sway elections and public opinion.
Chat-style search’s tendency to hallucinate is likely to accidentally produce medical misinformation.
Chatbot companions that offer emotional support have already been changed in arbitrary ways that left some users in serious emotional pain.
LLM-generated prose has already disrupted web forums and peer review processes, by flooding outlets with fake submissions.
The ladder is particularly worrisome because of the pace at which the problem is growing:
If misinformation (much harder to measure) grows at the same pace, we have a serious problem.
But the list I gave above is clearly just a start.
New concerns are emerging almost literally day,. Here are three examples that were forwarded to me, just in the last few days, ranging from one that relatively mild to that clearly more extreme.
The first is already starting to feel a bit familiar: gaslighting. But instead of a chatbot gaslighting its user, trying to persuade it that something untrue is true, a chatbot seems to have mislead a well-intention user (perhaps a student?) who was in turn trying to persuade a professor (who hadn’t consented to be part of an LLM experiment) to comment on a paper that the professor hadn’t actually written.
That one’s just a minor waste of time; things could be worse
The next example is a straight scam, made possible in new form by Bing,
The third is also disturbing:
And many of these attacks might of course be combined with advances in voice-cloning tech, which itself is already being applied to scamming, as discussed by the Washington Post on Sunday.
§
That’s a lot for one week.
Lately I have been asked to participate in a bunch of debates about whether LLMs will, on balance, be net positive or net negative. As I said in the most recent one (not yet aired), at the beginning of my remarks, the only intellectually honest answer is to say we don’t yet know. We don’t know how high the highs are going to be, and we don’t yet know how low the lows are going to be.
But one thing is clear: anybody who thinks we have come to grips with the unknown unknowns here is mistaken.
Update: The day after I posted the above, Tristan Harris posted this:
Gary Marcus (@garymarcus), scientist, bestselling author, and entrepreneur, is a skeptic about current AI but genuinely wants to see the best AI possible for the world—and still holds a tiny bit of optimism. Sign up to his Substack (free!), and listen to him on Ezra Klein. His most recent book, co-authored with Ernest Davis, Rebooting AI, is one of Forbes’s 7 Must Read Books in AI. Watch for his new podcast, Humans versus Machines, this Spring.
Many of these critiques might have been applied to the printing press, typewriters or paper: they allow humans to create problematic information and do things that are *already* against the law. (if they weren't against the law, then its questionable to complain that using AI makes them somehow more problematic).
Its unclear what your silly examples add to any attempt at pragmatic discussion other than being clickbait examples to try to get reader. They masquerade as if they were adding something to the debate but they are merely obvious potential examples of the class of issues that need to be addressed. They don't add anything to seriously considering the issue of if or how to address them other than seemingly being an attempt to ramp up moral panic porn.
re: the 1st example of someone asking about a paper that doesn't exist: yup, the software has glitches and it seems possible to educate all but truly blithering idiots that they should check facts. Thats a problem regardless of where they get information from. If anything this was a case where a fact was being checked, merely inefficiently and wasting a profs time.
In the real world information from any source can have glitches. If anything perhaps a higher level of glitches will teach people to be careful to evaluate information from multiple sources.
2nd example: yes, software can be used to scam people as its been able to for decades. Its against the law already: but I guess you wish to make it doubly against the law as if that'll help? Again, its useful to teach people to be careful with their credit card information. We can't child-proof/idiot-proof the whole world.
3rd: yes: just as a word processor can be used to create BDSM, etc. written porn. Or a printing press.
4th: again: So instead of a stranger saying "X has been in an accident and isn't conscious to talk", this made it slightly easier to dupe someone. Yup, people can be scammed, this made it a bit easier. Its still against the law already.
Often in the real world its difficult to judge the credibility of information, for instance a professor that doesn't bother to learn about the academic work regarding topics he comments on like regarding public choice theory or regulatory capture. Its what leads other professors to have a hard time taking a simplistic poorly reasoned argument from a poorly informed source seriously.
re: "Lately I have been asked to participate in a bunch of debates about whether LLMs will, on balance, be net positive or net negative. "
The same might be said of humans. Humans can create problematic content also, with or without tools. Puritans and religious zealots have been concerned about people being able to create pornography or print problematic ideas since the creation of writing, and then again when the printing press arose. Unfortunately some authoritarians tend to be concerned that they can't control each and every action of humans to ensure they do nothing wrong. Others resist that temptation, but see an excuse to give in to their desire to control others when some new tech comes along.
I predict that, before we answer the "Are LLMs a net-positive?" question, we will have stopped calling them LLMs. The AIs we ask this question of will only use LLM technology as their language module.