An epistemic clusterfuck in the making

Social media was bad. Adding AI into the mix could easily get a lot worse.

Epistemology is, roughly speaking, the study of how we know what know. In the tech world, people sometimes talk about the epistemic health of some systems, asking e.g., whether people getting the right information and evaluating the right way. An epistemic clusterfuck would be a pernicious system that systematically distorts reality, perhaps in ways that few people are aware of. Social media might be an example. But things may actually get worse.

The journalist Alec Kantrowitz just got a scoop from Elon Musk that made me a little sick to my stomach. Quoting in part…

The part that alarmed me the most was that at least in the near term, X has no plans to have the AI consult the original sources. Instead, all the synthesis will come from inside X’s own bubble:

Ouch. I use X a lot, probably more than I should, and I can tell you two things: there is very little culture of fact checking there, and very little culture of trying to rationally reflect on and resolve competing opinions, which inevitably leads to extremism. LLMs are not up to that job. LLMs confined to X tweets will be even less grounded in reality.

Automatically amplifying the already inherent tendency towards extremism without fact-checking or reasoned analysis is a recipe for chaos.

All Gary Marcus wants for Christmas is a healthy information ecosphere.