Detecting coordinated campaigns: text similarity, timing, networks, and the base-rate problem

Coordinated campaign detection is the set of methods used to find organized, fabricated or automated participation in comment systems, surveys and online platforms. The methods look for shared text, unusual timing, and groups of accounts that act alike. The 2017 FCC net neutrality record shows both their value and their limits: millions of fake comments were found, but one campaign generated unique text and staggered timestamps specifically to avoid detection.

In plain words

When many people send the same message because an advocacy group asked them to, that is normal democracy. When one company or one person sends millions of messages using stolen or invented names, that is fraud. Both can look alike at first glance.

Detection tools look for clues: messages that are nearly identical, bursts of submissions at odd times, and groups of accounts that always act together. Each clue can be faked. One campaign used software to write a different message for every fake name, and invented the times the messages were “sent” so they would look spread out.

There is also a math problem. If fake entries are rare, even a good detector flags many real people by mistake. So a flag should start a review, not end one.

Collective priorities records the scale of the FCC case. This page explains how detection works. Insincere survey respondents are covered in Opt-in panels.

Defined terms

• Mobilization. Organized encouragement of real people to participate, such as form letters from advocacy groups. Legitimate.
• Fabrication. Submissions in the names of people who did not make them, whether real (stolen identities) or invented.
• Automation. Software that submits at scale, with or without real identities.
• Sybil identities. Multiple identities controlled by one actor, a term from John Douceur’s 2002 paper “The Sybil Attack”.⁠⁶
• Near-duplicate. Two texts that are “roughly the same,” measured by resemblance.⁠³
• Shingle. A short overlapping sequence of words used to compare texts.
• Burst. A period in which some feature appears at sharply elevated frequency.⁠⁴
• Coordination network. A graph linking accounts that share “arbitrary behavioral traces”.⁠⁵
• Precision. Of the items a detector flags, the share that are truly fraudulent.

Evidence

What fabrication looked like at scale

The New York State Office of the Attorney General’s 2021 report found that the FCC’s 2017 net neutrality proceeding drew “more than 22 million comments, dwarfing the prior record of 3.7 million comments set by the prior FCC net neutrality proceeding in 2015,” and that “nearly 18 million” were fake.⁠¹ The report describes several distinct mechanisms:

• Lead generators fabricating consent. Firms paid to enroll consumers instead copied “old data ... they had compiled for other campaigns or purchased from other lead generators”.⁠¹
• Fake timestamps. “To conceal that the information had been copied without individuals’ consent, the companies generated false timestamps for each record,” and those timestamps “were generated at varying intervals over the course of the day, to avoid the appearance that they were artificial”.⁠¹
• Generated text. Aware that “a large volume of identical comments could attract scrutiny,” one company “used software to create unique text for each comment,” a template “akin to a Mad Libs” with “35 of these Mad Libs-style blank fields, each of which had between 2 and 30 words or phrases that could be used”; an executive overseeing the effort counted “[n]early 24 decillion permutations”.⁠¹
• Invented identities by one person. A 19-year-old college student filed “over 7.7 million pro-neutrality comments” using “fabricated names and addresses generated by software,” and the report found that “the FCC had few safeguards in place to detect or prevent millions of submissions from a single source”.⁠¹

The report also found that three of the lead generation firms had worked on “more than 100 other unrelated advocacy campaigns,” generating more than 1 million comments to the Environmental Protection Agency, the Bureau of Ocean Energy Management and the FCC, and “more than 3.5 million digital signatures for letters and petitions,” and that “nearly all” were fake.⁠¹ Fabrication is not a one-off; it is a commercial service.

Text similarity: resemblance and containment

Andrei Broder’s 1997 paper defined two measures for documents A and B, “resemblance r(A, B) and ... containment c(A, B),” which “capture well the informal notions of ‘roughly the same’ and ‘roughly contained,’” by reducing comparison “to set intersection problems that can be easily evaluated by a process of random sampling”.⁠³ With S(A) the set of word shingles in A:

r(A, B) = |S(A) intersect S(B)| / |S(A) union S(B)|

c(A, B) = |S(A) intersect S(B)| / |S(A)|

Resemblance finds form letters. Containment finds a form letter with a personal sentence added. Sampling methods descended from this work make it practical to compare millions of texts.

The FCC template shows the limit. If each of 35 slots had even 2 options, the template could produce 2^35, about 34 billion, distinct texts; with up to 30 options per slot, far more. Pairwise resemblance between two outputs may be modest. Yet every output shares the template’s fixed skeleton in the same order. Detection therefore shifts from “are these two texts alike?” to “do many texts share an unusual structure?”, for example by aligning texts to find a common frame with variable slots. That is a harder, more model-dependent inference.

Timing: bursts and their imitations

Jon Kleinberg’s 2002 work modeled document streams “using an infinite-state automaton, in which bursts appear naturally as state transitions,” yielding algorithms that are “highly efficient” and “a nested representation of the set of bursts”.⁠⁴ Applied to participation, burst detection flags sudden spikes in submissions sharing a feature: a phrase, a referring link, a district.

Bursts are ambiguous. A televised hearing, a news story, or a school sending home a notice all cause legitimate bursts. And the FCC fabricators deliberately smoothed their timestamps across the day,⁠¹ the opposite of a burst. A system that trusts client-reported times will be fooled; server-side arrival times are harder, though not impossible, to fake.

Networks: coordination as shared behavior

Pacheco and colleagues, at the 2021 International AAAI Conference on Web and Social Media, introduced “a general, unsupervised network-based methodology to uncover groups of accounts that are likely coordinated,” which “constructs coordination networks based on arbitrary behavioral traces shared among accounts”.⁠⁵ In their case studies they detected coordinated accounts “by examining their identities, images, hashtag sequences, retweets, or temporal patterns”.⁠⁵ The general recipe:

1. Build a bipartite graph linking accounts to traces (a text fingerprint, a device signature, a time window, a referral source).
2. Project it onto accounts, weighting each pair by how many unusual traces they share.
3. Keep edges above a threshold and examine dense clusters.

The method is honest about what it finds: coordination, not fraud. A union local, a church group and a botnet can all form dense clusters. Stefano Cresci’s review of a decade of social bot detection in Communications of the ACM frames the task as an ongoing contest, “tracing trends in detection strategies and key suggestions on how to win the fight”,⁠⁷ which is a reminder that any published detector invites adaptation.

Survey responses: insincerity has a direction

In surveys, the analog is the insincere respondent. Kennedy and colleagues’ study of six online sample sources analyzed 375,834 open-ended answers because closed-ended answers and paradata such as IP addresses “are relatively easy for bad actors to fake”.⁠² They found insincere respondents “tended to select positive answer choices, introducing a small, systematic bias”.⁠² Open-ended text is a stronger signal because nonsense, off-topic or copied answers are hard to produce convincingly at scale. Even so, generated text from modern language models may erode that advantage, an open question for any method that relies on text quality.

The base-rate problem

Detection accuracy depends on how rare fraud is. Using Bayes’ rule, the precision of a flag is:

precision = (sensitivity * prevalence) / ( sensitivity * prevalence + false_positive_rate * (1-prevalence) )

Hypothetical example. A detector catches 95% of fabricated entries and wrongly flags 5% of genuine ones. If 2% of entries are fabricated, precision is (0.95 × 0.02) / (0.95 × 0.02 + 0.05 × 0.98) = 0.019 / 0.068, about 28%. Nearly three of every four flagged participants are genuine. If fabricated entries are 80% of the total, close to the FCC docket’s nearly 18 million of more than 22 million,⁠¹ precision rises to about 99%. The same detector is excellent in a flood and mostly wrong in a trickle.

This arithmetic is why automatic deletion of flagged participation is dangerous in ordinary conditions, and why flag rates should be reported alongside the evidence for each.

Competing interpretations

“Open intake is the democratic default; detection is gatekeeping.” On this view, any filter risks excluding real people, especially those who use shared devices, public libraries or assistance from organizers. The base-rate arithmetic supports caution.

“Without detection, volume wins.” The FCC record.⁠¹ shows that an open system without safeguards can be dominated by fabricated volume on both sides of a question. On this view, a participation system without detection invites the result it fears.

“Verify identity at the door instead.” Stronger identity checks prevent Sybil identities more reliably than downstream detection, but they also exclude people and create sensitive records. The tradeoff is discussed in Identity and registration and Does a fingerprint prove eligibility?.

“Mobilization should be visible, not suppressed.” A middle view: identical letters from real people are legitimate but are also one organized voice. Reporting them as a cluster, with their count, informs readers without deleting anyone.

Application to this framework

These are design proposals for My Vote Forward, not results.

1. Prevent first, detect second. One person, one active priority record, enforced at registration (see Identity and registration), reduces the need for statistical detection. Detection is a second layer for what prevention misses.
2. Use server-side evidence. Arrival times, rate limits and submission patterns recorded by the server, not timestamps supplied by clients, because fabricated timestamps were central to the FCC case⁠¹
3. Report clusters, do not delete them. Releases show how much participation came through identified coordinated channels (shared referral links, identical free text, synchronized timing), alongside results with and without those clusters. Readers see whether a cluster changed the order.
4. Review flags with humans and log outcomes. Each flag carries its evidence; a reviewer records the decision and reason; aggregate flag and reversal rates are published, so precision can be estimated over time.
5. Do not publish detector thresholds in operational detail. Publish the categories of signals and the review process, but not exact thresholds, following Cresci’s observation that detection is an adversarial contest.⁠⁷ This is a tension with transparency and is listed as unresolved.
6. No payments. Not paying participants removes the commercial motive that drove lead-generator fraud,⁠¹ and survey insincerity⁠² though not political motives.

Unresolved questions

• How much detail about detection methods can be published before it helps adversaries, and who decides?
• How should a release treat a large, genuine but coordinated cluster that changes the published order? Report both versions, weight it, or neither?
• Will generated text from language models make open-ended text checks ineffective, as they made Mad Libs templates look obsolete?
• What appeal process is proportionate for a nonbinding priority exercise?
• Should coordinated-campaign reports name the organizing group when known, and what are the risks of doing so?