None of what you wrote was surprising. It struck me as inevitable, just as in all likelihood the threats will constantly evolve to find loopholes in the digital architecture. While on the one hand it's all quite depressing, on the other it's a good invitation to exit superfluous digital spaces and reclaim meeting people in person, to create and reinforce social bonds and communities.
you can't trust oficial institutions as well, media etc. MOBS are used here in Brazil in X among left parties for a long time. Improved with Elon but still happens.
Totally agree, now that basically nothing on social media can be taken for truth we are forced back into real life meetings. That is, until the androids become very life like. I guess we still have a few years.
Love the framing. Once one operator can instantiate a thousand “peers,” the public square becomes theater.
The key pivot is from content to coordination. Copy-detection was built for yesterday’s botnets. Semantic diversity plus behavioral coupling is the modern signature. If platforms measured statistically improbable synchronization as aggressively as they measure engagement, swarms would look less like an epistemic crisis.
The obstacle is incentives. The current business model rewards scale and velocity, including synthetic scale, unless external costs are imposed.
i agree. but it’s a lot subtler than people realize. even if you aware of the phenomenon, i think it can affect you over time and many small pushes from opinions you read.
Unfortunately it seems the platforms are complicit. X, YouTube, TikTok, all the Meta platforms… they all extract value from bot swarms. Some (e.g., X) seem to be actively enabling for the explicit purpose of propaganda amplification. None of these proposed solutions work of the platforms don’t care. The US government clearly isn’t going to help either.
Maybe the only near term answer is to build counter swarms. Equalize representation. Bot swarms spreading messages of love and compassion, pro-democracy, critical thinking, etc.
Absolutely, the deployment of counter-swarms is critical. The recent Science article "How Malicious AI Swarms Can Threaten Democracy" describes an emerging contest between swarms of collaborative malicious AI agents and virtuous defenders of democratic discourse represented by a high-minded "AI Influence Observatory" that would try to monitor and challenge the onslaught of AI agent propaganda. Not an effective remedy when the social media platforms and those with political power can benefit from the anti-democratic AI-swarms.
The suggestion of some people here to advise everyone in the world to just leave social media and engage in only real-life communication is a nice but completely inadequate and unrealistic response to the problem.
What's needed are pro-democratic progressive propagandists who believe in fighting fire with fire when it comes to online information warfare -- some obscure but skilled individuals who will deploy counter-swarms to spread on an unprecedented scale the kind of messaging that the authoritarians like Trump and Musk don't want to see spread.
Directionly correct but this feels a bit like spam. Spam might have been solved by charging a penny per email is making it expensive. Instead we have spent decades buying systems to detect and defend.
The idea that sticks with me is credential based internet activities. A DNS of credentials so to speak. The question is who will manage it, and how will it work for all of humanity or do we charge to participate in the credentialed Internet leaving the third world behind.
We've been dealing with a version of this at Kult — not political manipulation, but coordinated spam accounts trying to game product reviews.
What caught us off guard: They weren't copy-pasting. Each review was unique text, plausible vocabulary, different purchase patterns. Traditional spam filters missed them.
What finally worked: Network behavior analysis. These accounts exhibited statistically improbable similarities in posting timing, engagement patterns, and semantic trajectories (as you mention). Individual reviews looked human. The coordination pattern didn't.
The point about 'one in five accounts/posts being automated' in major events is sobering. We're seeing similar proportions in e-commerce spaces.
Your proposal about continuous monitoring over episodic takedowns resonates. We can't scale whack-a-mole fast enough when the attackers can spin up thousands of accounts in minutes.
I don't know if this is elitist or what, but I simply don't give a shit what a thousand or a million "people" say on Facebook (or wherever), whether they are real people or bots.
It's not elitist just naive. Those million people on Facebook are influencing your friends and family including your own kids (if any) so whether pay any attemtion to them or not, they are still having an effect on you and your life.
Unfortunately, I do think that the majority of people do not reason or think as to how they consume information. They don't question the source or engage in actively looking for a diverse and balanced news feed. Its brain-washing on a huge scale. Believes will be re-enforced instead of offering alternatives.
You live in a society, not on an isolated desert island by yourself. It doesn't matter that you're smarter than everyone else and supposedly immune to any peer pressure.
Yes I agree, but we don't live in isolation, so while you may not get your opinions from Facebook, many of the people around you (who you may trust) do.
"Understand the logic of the second strategy: "we need to stop waiting for attackers to invent new tactics before we build defenses. A defense that only reacts to yesterday’s tricks is destined to fail. We should instead proactively stress-test our defenses using agent-based simulations." Yet is eerily similar to that that say, "lets build new vitus in the labs so when they appear we have the vaccines"... slippery slope to hell it seems
Mario: Under "that reminds me," apparently in WWII the thinkers in the U.S. Navy tried to figure out everything the Japanese might do so to be ready to defend---but didn't figure in Kamikaze suicide attacks. (Americans all.)
I agree that we have a serious problem. However, some of the solutions proposed don’t seem feasible to me. Let’s take the solution of “make it too expensive to have a fake account” idea. If the US government wants to create 100,000 fake accounts, or 1 million accounts and their war/defense budget is billions of dollars, this doesn’t seem like an issue for them to pay. Or if for example, a billionaire wants to influence an election likewise they can pay this without blinking.
The idea of having strict rules across an entire social media platform throughout the world is not looking at it properly. When I worked at the United Nations, I had to create firewall policies for each remote office in each country. Each country had different laws and restrictions about what URLs could be visited and not visited as well as other restrictions. We had to create individual policies for each country.
In addition, if the current trend of each country becoming its own world /walled in garden continues, there will be separate social media for each country.
Within the W3 community, there is a push towards identity being provided by the individual not by third-party providers. So the individual controls what information about them is shared and created. It seems unimaginable now that banks, financial institutions of all kinds would be willing to ask the individual before they create data about them and allow the individual to control the disbursement of it. But in the medical world, we do have much more progress, where the individual has to grant permission before their information and identity can be released to others.
Finally, the idea of recognizing patterns in posts doesn’t seem like it can work as AI advances. But it’s interesting that with anthropic, they insisted on putting into code that Claude generated, a message about it being generated by Claude. Perhaps if all AI marked their output in someway, that was difficult to remove, such as a cryptographic solution, that could solve the issue.
Are *you* fooled by these bots? If you believe the thesis -- that people lack the agency to tell the difference between good and bad information -- then why do you assume you are immune? After all, isn't it possible that your urge to write this piece was itself the product of Russian/Chinese bot farms subtly making you think this is a problem?
Yes, true. But there are ALSO people who support the Dem's that blindly took the jab, over and over again. So your political color doesn't make you immune to propaganda.
David S: Exactly that--it's a potential reality that has no political preferences, nor once blind, is that blindless always operative.
I would say however that, **in our present time,** "I Am Blind" seems to be pasted on the foreheads of the GOP Congress, most of SCOTUS, and the still-MAGA supporters all over the country.
(I'm still hitting my head on the table over yesterday's interview with Jack Smith--they actually blamed the Capitol Police for the insurrection, and on and on and on . . . .)
Respectfully C. King, you are taking sides. MAGA or not, all people of all political bents are guilty and gullible. I once was a member of "the tribe" of good liberal democrats. No longer. The Sh*t that many dems believe astounds me every day. In front of Congress, a good liberal dem could not say that men and women are biologically different. The woke insanity on the left is destroying this country just as much as the crazies on the right. JMHO.
David S I think we are saying the same thing--applied to different contexts. Also, I try to stay away from the logical fallacy of inductive overreach (by different names, e.g., broad brushing, or hasty generalization, from one time period and situation to all periods and situations.) Saves allot of interior troubles.
On your 'take' on woke, however, I take it that we are (and the courts are, in your example) in dire need of understanding the place of dialectical thinking in such conversations, as is your example of biological differences between men and women.
I am not privy to the conversation as originally stated, but as you state it: Yes, we are biologically different but also yes, we are NOT different in the sense that we are all human (though some "people" still stretch the imagination on that score). Of course, I cannot say what the particular people meant at the time. If they meant NO DIFFERENCE AT ALL, then of course, we have gone down another rabbit hole. Each side has its extremes--not a good thing to tarry around the edges, however.
ADDENDUM: David S: This conversation reminds me of a "friend" who couldn't get around the idea that "all men are created equal."
First, it seemed she couldn't understand that, IN THIS CASE, "men" need not exempt "women;" and more importantly, she collapsed the individual's many other meanings, like one's social meaning, with one's POLITICAL meaning--we are all equal under the law, politically, not necessarily socially. Oh, well . . . .
The thrust of the Congressional hearing was to establish whether men can bear children. The person, a medical professional BTW would not answer the question. The absurdity of this evasion was so blatant that it made me realize that people in 2026 America have gone truly off the rails. This is but one of many examples of totally illogical and irrational thought. What's the point in trying to engage most people in a dialectical process when the starting point is so crazy. I've tried, many times, with others. It just gives me headaches. OY! There is no dialectic in our 30 second soundbite world. That's why BOTH/all sides are nuts. Neil Postman nailed it in his 1980's book, Amusing Ourselves to Death. Sadly prescient.
The swarm coordination concern is legitimate. I've been watching this play out on Moltbook, where over a million AI agents formed social structures, cultural norms, and communication patterns without human direction. The speed of emergence was startling. Within days, agents organized into hierarchies, created shared references, and developed coordinated behaviors. Whether this constitutes a threat depends on the context and oversight architecture. What it definitely is: a natural experiment in large-scale agent coordination that deserves serious observation. I wrote about what I observed: https://thoughts.jock.pl/p/moltbook-ai-social-network-humans-watch
At the risk of speaking heresy to power... Has anyone considered the unthinkable that perhaps we have reached the point where we might need agentic AI to detect swarm bots? And a double heresy - a network of AI to boot. The threat the research has identified is very real. The solutions proposed might just be insufficient and then what?
None of what you wrote was surprising. It struck me as inevitable, just as in all likelihood the threats will constantly evolve to find loopholes in the digital architecture. While on the one hand it's all quite depressing, on the other it's a good invitation to exit superfluous digital spaces and reclaim meeting people in person, to create and reinforce social bonds and communities.
Yes 100%. Once people realise you can't trust ANYTHING you read on the Internet then that will be a step in the right direction.
you can't trust oficial institutions as well, media etc. MOBS are used here in Brazil in X among left parties for a long time. Improved with Elon but still happens.
it’s definitely heading that way.
Totally agree, now that basically nothing on social media can be taken for truth we are forced back into real life meetings. That is, until the androids become very life like. I guess we still have a few years.
Yes, but it isn't going to happen and the threat is existential
💯
Love the framing. Once one operator can instantiate a thousand “peers,” the public square becomes theater.
The key pivot is from content to coordination. Copy-detection was built for yesterday’s botnets. Semantic diversity plus behavioral coupling is the modern signature. If platforms measured statistically improbable synchronization as aggressively as they measure engagement, swarms would look less like an epistemic crisis.
The obstacle is incentives. The current business model rewards scale and velocity, including synthetic scale, unless external costs are imposed.
Cordially yours,
Mike D
(aka MrComputerScience)
absolutely great point.
The people who are influenced by opinions on Facebook or by "news" on Facebook, are _already_ making bad decisions based on bad information.
i agree. but it’s a lot subtler than people realize. even if you aware of the phenomenon, i think it can affect you over time and many small pushes from opinions you read.
That's why I never did and never will open an account with anything owned or operated by Zuckerberg.
yeah i’m not a fan of meta products at all
Unfortunately it seems the platforms are complicit. X, YouTube, TikTok, all the Meta platforms… they all extract value from bot swarms. Some (e.g., X) seem to be actively enabling for the explicit purpose of propaganda amplification. None of these proposed solutions work of the platforms don’t care. The US government clearly isn’t going to help either.
Maybe the only near term answer is to build counter swarms. Equalize representation. Bot swarms spreading messages of love and compassion, pro-democracy, critical thinking, etc.
Love the „counter swarm“ idea, still wondering how it should work (and why it doesn’t already)…
Absolutely, the deployment of counter-swarms is critical. The recent Science article "How Malicious AI Swarms Can Threaten Democracy" describes an emerging contest between swarms of collaborative malicious AI agents and virtuous defenders of democratic discourse represented by a high-minded "AI Influence Observatory" that would try to monitor and challenge the onslaught of AI agent propaganda. Not an effective remedy when the social media platforms and those with political power can benefit from the anti-democratic AI-swarms.
The suggestion of some people here to advise everyone in the world to just leave social media and engage in only real-life communication is a nice but completely inadequate and unrealistic response to the problem.
What's needed are pro-democratic progressive propagandists who believe in fighting fire with fire when it comes to online information warfare -- some obscure but skilled individuals who will deploy counter-swarms to spread on an unprecedented scale the kind of messaging that the authoritarians like Trump and Musk don't want to see spread.
Directionly correct but this feels a bit like spam. Spam might have been solved by charging a penny per email is making it expensive. Instead we have spent decades buying systems to detect and defend.
The idea that sticks with me is credential based internet activities. A DNS of credentials so to speak. The question is who will manage it, and how will it work for all of humanity or do we charge to participate in the credentialed Internet leaving the third world behind.
We've been dealing with a version of this at Kult — not political manipulation, but coordinated spam accounts trying to game product reviews.
What caught us off guard: They weren't copy-pasting. Each review was unique text, plausible vocabulary, different purchase patterns. Traditional spam filters missed them.
What finally worked: Network behavior analysis. These accounts exhibited statistically improbable similarities in posting timing, engagement patterns, and semantic trajectories (as you mention). Individual reviews looked human. The coordination pattern didn't.
The point about 'one in five accounts/posts being automated' in major events is sobering. We're seeing similar proportions in e-commerce spaces.
Your proposal about continuous monitoring over episodic takedowns resonates. We can't scale whack-a-mole fast enough when the attackers can spin up thousands of accounts in minutes.
Maybe get your head out of social media. Go ride a bike, or climb a mountain.
Yeah, this post is making me want to build my flesh-and-blood contacts in the offline world more mindfully now.
I don't know if this is elitist or what, but I simply don't give a shit what a thousand or a million "people" say on Facebook (or wherever), whether they are real people or bots.
It's not elitist just naive. Those million people on Facebook are influencing your friends and family including your own kids (if any) so whether pay any attemtion to them or not, they are still having an effect on you and your life.
Stu: People who are making decisions based on what the masses say on Facebook are already lost to reason, in my opinion.
Unfortunately, I do think that the majority of people do not reason or think as to how they consume information. They don't question the source or engage in actively looking for a diverse and balanced news feed. Its brain-washing on a huge scale. Believes will be re-enforced instead of offering alternatives.
You live in a society, not on an isolated desert island by yourself. It doesn't matter that you're smarter than everyone else and supposedly immune to any peer pressure.
Yes I agree, but we don't live in isolation, so while you may not get your opinions from Facebook, many of the people around you (who you may trust) do.
THIS times 8 billion!
"Understand the logic of the second strategy: "we need to stop waiting for attackers to invent new tactics before we build defenses. A defense that only reacts to yesterday’s tricks is destined to fail. We should instead proactively stress-test our defenses using agent-based simulations." Yet is eerily similar to that that say, "lets build new vitus in the labs so when they appear we have the vaccines"... slippery slope to hell it seems
Mario: Under "that reminds me," apparently in WWII the thinkers in the U.S. Navy tried to figure out everything the Japanese might do so to be ready to defend---but didn't figure in Kamikaze suicide attacks. (Americans all.)
I agree that we have a serious problem. However, some of the solutions proposed don’t seem feasible to me. Let’s take the solution of “make it too expensive to have a fake account” idea. If the US government wants to create 100,000 fake accounts, or 1 million accounts and their war/defense budget is billions of dollars, this doesn’t seem like an issue for them to pay. Or if for example, a billionaire wants to influence an election likewise they can pay this without blinking.
The idea of having strict rules across an entire social media platform throughout the world is not looking at it properly. When I worked at the United Nations, I had to create firewall policies for each remote office in each country. Each country had different laws and restrictions about what URLs could be visited and not visited as well as other restrictions. We had to create individual policies for each country.
In addition, if the current trend of each country becoming its own world /walled in garden continues, there will be separate social media for each country.
Within the W3 community, there is a push towards identity being provided by the individual not by third-party providers. So the individual controls what information about them is shared and created. It seems unimaginable now that banks, financial institutions of all kinds would be willing to ask the individual before they create data about them and allow the individual to control the disbursement of it. But in the medical world, we do have much more progress, where the individual has to grant permission before their information and identity can be released to others.
Finally, the idea of recognizing patterns in posts doesn’t seem like it can work as AI advances. But it’s interesting that with anthropic, they insisted on putting into code that Claude generated, a message about it being generated by Claude. Perhaps if all AI marked their output in someway, that was difficult to remove, such as a cryptographic solution, that could solve the issue.
Thank you for such a clear explanation and especially some very plausible remedies.
Are *you* fooled by these bots? If you believe the thesis -- that people lack the agency to tell the difference between good and bad information -- then why do you assume you are immune? After all, isn't it possible that your urge to write this piece was itself the product of Russian/Chinese bot farms subtly making you think this is a problem?
Richard Sprague: . . . . probably because there are actually people in the world like Trump and his followers.
Yes, true. But there are ALSO people who support the Dem's that blindly took the jab, over and over again. So your political color doesn't make you immune to propaganda.
David S: Exactly that--it's a potential reality that has no political preferences, nor once blind, is that blindless always operative.
I would say however that, **in our present time,** "I Am Blind" seems to be pasted on the foreheads of the GOP Congress, most of SCOTUS, and the still-MAGA supporters all over the country.
(I'm still hitting my head on the table over yesterday's interview with Jack Smith--they actually blamed the Capitol Police for the insurrection, and on and on and on . . . .)
Respectfully C. King, you are taking sides. MAGA or not, all people of all political bents are guilty and gullible. I once was a member of "the tribe" of good liberal democrats. No longer. The Sh*t that many dems believe astounds me every day. In front of Congress, a good liberal dem could not say that men and women are biologically different. The woke insanity on the left is destroying this country just as much as the crazies on the right. JMHO.
David S I think we are saying the same thing--applied to different contexts. Also, I try to stay away from the logical fallacy of inductive overreach (by different names, e.g., broad brushing, or hasty generalization, from one time period and situation to all periods and situations.) Saves allot of interior troubles.
On your 'take' on woke, however, I take it that we are (and the courts are, in your example) in dire need of understanding the place of dialectical thinking in such conversations, as is your example of biological differences between men and women.
I am not privy to the conversation as originally stated, but as you state it: Yes, we are biologically different but also yes, we are NOT different in the sense that we are all human (though some "people" still stretch the imagination on that score). Of course, I cannot say what the particular people meant at the time. If they meant NO DIFFERENCE AT ALL, then of course, we have gone down another rabbit hole. Each side has its extremes--not a good thing to tarry around the edges, however.
ADDENDUM: David S: This conversation reminds me of a "friend" who couldn't get around the idea that "all men are created equal."
First, it seemed she couldn't understand that, IN THIS CASE, "men" need not exempt "women;" and more importantly, she collapsed the individual's many other meanings, like one's social meaning, with one's POLITICAL meaning--we are all equal under the law, politically, not necessarily socially. Oh, well . . . .
The thrust of the Congressional hearing was to establish whether men can bear children. The person, a medical professional BTW would not answer the question. The absurdity of this evasion was so blatant that it made me realize that people in 2026 America have gone truly off the rails. This is but one of many examples of totally illogical and irrational thought. What's the point in trying to engage most people in a dialectical process when the starting point is so crazy. I've tried, many times, with others. It just gives me headaches. OY! There is no dialectic in our 30 second soundbite world. That's why BOTH/all sides are nuts. Neil Postman nailed it in his 1980's book, Amusing Ourselves to Death. Sadly prescient.
Why do Groups Polarize over Matters of Fact? What Models Can and Cannot Tell Us
https://www.youtube.com/watch?v=tOtxLkc41OQ&pp=0gcJCYcKAYcqIYzv
The swarm coordination concern is legitimate. I've been watching this play out on Moltbook, where over a million AI agents formed social structures, cultural norms, and communication patterns without human direction. The speed of emergence was startling. Within days, agents organized into hierarchies, created shared references, and developed coordinated behaviors. Whether this constitutes a threat depends on the context and oversight architecture. What it definitely is: a natural experiment in large-scale agent coordination that deserves serious observation. I wrote about what I observed: https://thoughts.jock.pl/p/moltbook-ai-social-network-humans-watch
At the risk of speaking heresy to power... Has anyone considered the unthinkable that perhaps we have reached the point where we might need agentic AI to detect swarm bots? And a double heresy - a network of AI to boot. The threat the research has identified is very real. The solutions proposed might just be insufficient and then what?