Algorithmic Influence is a Population Vulnerability

In February I wrote that a population conditioned by recommendation algorithms could be injected with adversarial narratives in the same way a language model can be injected with hidden instructions, and that actors who understood what the algorithms rewarded could feed them material to move people toward an outcome. I referred to this a cognitive prompt injection. This week the New York Times reported the prediction came true. 

The reporting describes several campaigns, run by states in some cases and by private firms in others, that combined freely downloadable language models with software agents. The safeguards that would normally stop a model from opening a fake account or coordinating a messaging operation were switched off by the operators. Hundreds of agents then handled every stage on their own: registering the accounts across four major platforms, writing the posts, tagging journalists and politicians, timing the whole thing. One network reportedly gathered close to eighty thousand followers before anyone found out. A firm involved in a separate campaign said in an interview that it had purchased ten thousand social media accounts for agents to operate, of which roughly a thousand went live, and that the model chose the topics and the phrasing itself.

I won’t name the countries or the companies here (visit the link for those details) because the attribution is the least interesting part of the story and it is the part that will make people argue about the wrong thing.

The major change here is the labor.

Influence Operations No Longer Require Staff

An influence operation used to be a staffing problem. Someone had to sit in a building and open the accounts, write in a second language convincingly enough, watch which posts moved and which died, and coordinate the timing across platforms with other people doing the same work. That requirement functioned as a filter. It meant the capability belonged to organizations that could afford a payroll, tolerate the operational security risk of employing dozens of people who knew what they were doing, and absorb the cost of being caught. Agents remove the filter. The work that used to require a floor of staff now requires a downloaded model, some hardware, and a person willing to oversee. The pool of actors who can run a credible campaign expands from states with budgets to anyone with a grievance and a modest amount of money.

There is a second consequence that follows from the choice of model. Systems whose weights are published can be run on private hardware, which means no vendor sits above the operation with the ability to shut it off. The remedy people assume exists, a company revoking access when it notices misuse, does not apply. There is nothing to revoke.

The common version of the argument from here is that the public has become gullible, that test scores fell and attention spans shortened and the result is a population that believes whatever it is told. I do not think that is right, and it misses the more entrenched mechanism.

Recommendation systems do not homogenize a population. They fragment it, then homogenize inside each fragment. What you see next is determined by what the system has already inferred about you, and what you see next confirms the inference, which sharpens the next delivery. Each person ends up inside a loop that is internally consistent and externally narrow, and the consistency is what makes it feel like judgment rather than supplied. A frame injected into that loop does not have to persuade anyone of anything new. It has to match what the system already knows you are willing to hear.

The evaluative work that would catch this is the same work being handed over elsewhere. Gerlich's 2025 study found measurable declines in critical thinking associated with heavy AI use, steepest in the people who adopted earliest. I have called the professional version of this analytic atrophy, the erosion of independent analysis when the analysis itself, rather than the retrieval, gets offloaded. The population being targeted by cheap autonomous campaigns is the same population outsourcing the step where an injected frame would get caught. That is a claim about a practiced skill going unpracticed, which is more nuanced than anything about literacy scores.

Then there is the part of this that has no adversary in it at all.

In July, researchers at the Oxford Internet Institute and the Hasso Plattner Institute published a study showing that when large language models were asked to improve human-written posts on contested topics, they changed the position the posts expressed, and they did it while under explicit instruction to preserve the original meaning. Different models from different providers tilted in similar directions. Using network data from X and Facebook, the authors then simulated how small edits of that kind propagate, and found that the shifts accumulate across a network rather than cancelling out. 

The researchers rebuilt a platform feature that explains posts to readers, found it treated one side of the abortion debate more favorably than the other, and traced the imbalance by removing the platform's instructions one at a time until they isolated a single line telling the model to challenge mainstream narratives where necessary. Not the training data. Not the weights. One sentence written by a product team.

Put that next to what Sourati, Ziabari and Dehghani established in Trends in Cognitive Sciences this year, which is that people using these tools rarely steer the output. They select from the continuations the model offers and accept the one that is close enough to what they were going to say. Jakesch and colleagues had already shown, in 2023, that co-writing with an opinionated model shifted participants' own stated attitudes afterward, and that they did not notice.

The two findings interlock. If users are selecting rather than steering, and what is offered carries a direction, then the direction transfers to the user's published opinion and arrives with her name on it. No operation is required. No adversary has to be involved. The tilt is a property of the writing assistant and it travels through people who believe they are expressing themselves, which is a harder problem than a foreign campaign, because there is nobody to sanction and nothing to take down.

The commercial layer is arriving at the same time. The Conversation reported this week on advertising moving into AI assistants, with sponsored material appearing inside the generated answer rather than beside it, and with the same real-time bidding and behavioral micro-targeting that runs display advertising.  Brin and Page warned in their original paper on Google that advertising funding would bias search away from the reader and toward the buyer, and search advertising still had the decency to sit in a labeled box next to the results. Sponsored influence inside a generated answer has no box. The answer is one thing, delivered once, to one person.

The Defenses Were Built for a Public Artifact

Which brings the three vectors to the same place, and to the reason why the existing defenses won’t hold to these new campaigns.

Fact-checking, media literacy, platform takedown and vendor intervention were all built for a public artifact, a claim that is visible, persistent, attributable and available for someone other than its target to examine. Each of these recent three developments defeats one of those properties. Open weights defeat intervention, because there is no operator to call. Autonomous agents defeat attribution, because the operation looks like organic engagement and the human who started it is not in the loop. Personalization defeats examination, because a claim delivered privately to one reader and gone when she closes the window never enters the public record where a fact-checker could reach it. A fabricated allegation made to a single voter in a private conversation is not checkable in principle, not because the institutions are slow.

Freedom of thought has generally been treated as a settled right, protected because nobody could reach inside a person's head to violate it. The reach now exists, it costs very little, and the people using it include governments, companies and, increasingly, whoever else is willing to pay. The question of who is doing your thinking stops being a philosophical one at the moment the answer becomes purchasable.

---

Timothy Cook is Director of The Cognitive Privacy Project and author of the "Algorithmic Mind" column at Psychology Today. He is Securiti Certified in AI Security & Governance.

Contact: timothy@cognitiveprivacyproject.org Web: cognitiveprivacyproject.org

© 2026 Timothy Cook / The Cognitive Privacy Project. All rights reserved.Licensed under CC BY-NC-ND 4.0. You may share this work with attribution. Commercial use and derivatives require written permission.