Small-group suppression: when a published number is too small to be safe or too small to be true

Small-group suppression is the practice of withholding published statistics for groups too small to report safely or reliably. Federal statistical agencies use explicit rules for both reasons: the National Center for Health Statistics suppresses proportions whose confidence intervals are too wide, and the National Center for Education Statistics requires at least three unweighted cases per table cell when data are not otherwise protected. A minimum count is necessary but not sufficient, because suppressed cells can often be recovered by subtraction.

In plain words

There are two reasons not to publish a number about a small group.

The first is that the number may be wrong by chance. If 3 of 10 people in a neighborhood chose “schools,” that 30% could easily be 10% or 60% with ten different people. Publishing it as a fact would mislead.

The second is that the number may reveal someone. If a table shows that 1 person in a small precinct ranked “immigration” first, neighbors may guess who.

Hiding a number is not as simple as leaving a blank. If a table shows the total and every group but one, anyone can subtract to fill in the blank. And when results are published again next month, comparing the two releases can reveal what changed.

Hiding numbers also has a cost. Small communities can disappear from the record. This page explains the rules agencies use, why they work partly, and how a civic platform could apply them honestly. The noise-based alternative is in Differential privacy.

Defined terms

• Cell. One number in a published table, such as the count of participants in one district who ranked one subject first.
• Primary suppression. Withholding a cell because it fails a rule itself.
• Complementary (secondary) suppression. Withholding additional cells so that primary suppressed cells cannot be computed by subtraction.
• Differencing attack. Recovering a withheld value by subtracting published values, within one table, across tables, or across releases.
• Reliability suppression. Withholding an estimate because its uncertainty is too large, regardless of privacy.
• Disclosure suppression. Withholding a cell to protect individuals, regardless of its precision.
• k-anonymity. A property of a released table in which each person’s record “cannot be distinguished from at least k-1 individuals” on identifying attributes.⁠⁴
• Homogeneity attack. Learning a sensitive value because everyone in an indistinguishable group shares it.⁠⁵
• Recategorization (coarsening). Merging categories, such as age bands or neighboring areas, until cells are large enough.

Evidence

Reliability: the NCHS standards for proportions

The National Center for Health Statistics adopted presentation standards for proportions in 2017, written by its Data Suppression Workgroup.⁠¹ The rules are layered:

• Proportions “should be based on a minimum denominator sample size and effective denominator sample size (when applicable) of 30”.⁠¹
• A 95% two-sided confidence interval is computed “using the Clopper-Pearson method, or the Korn-Graubard method for complex surveys”.⁠¹
• If “the absolute confidence interval width is greater than or equal to 0.30, then the proportion should be suppressed”.⁠¹
• If the width is between 0.05 and 0.30 and “the relative confidence interval width is more than 130%, then the proportion should be suppressed”.⁠¹
• Estimates from complex surveys with “degrees of freedom ... fewer than 8” are flagged for review.⁠¹

The standards also note that observing zero events “can provide important information,” so such estimates are flagged for review rather than automatically hidden.⁠¹

The width rule does more work than the count rule. The following are computed Clopper-Pearson intervals, shown as worked arithmetic:

A group of 30 meets the minimum count yet fails the width test at a 50% share. Under these standards, a proportion for a group of 30 is rarely publishable. That is a useful calibration for any civic platform tempted to report precinct shares from a few dozen participants.

Geography thresholds: the ACS approach

The American Community Survey handles reliability through publication thresholds by geography. Its 1-year estimates cover “areas with populations of 65,000+,” and only the 5-year estimates cover “all areas”.⁠³ Small places are not suppressed cell by cell; they are reported only after enough years of data accumulate. The trade is timeliness: a small town’s figures describe a five-year window.

Disclosure: the NCES minimum-cell rule

The National Center for Education Statistics’ statistical standards take a different, privacy-based approach. When a survey program does not apply perturbation to a file, “any table with a cell with 1 or 2 unweighted cases must be recategorized to insure that each cell in the table has at least 3 unweighted cases,” and the rule “excludes table cells with zero cases because there are no data to protect in the cell”.⁠² The standard’s example: a table of principal salaries by race and experience with only two Asian respondents with more than 20 years of experience must combine categories “until all cells have either at least 3 unweighted cases or no unweighted cases”.⁠² The same standards allow files that have undergone approved confidentiality edits to “be used to produce tables without confidentiality concerns over minimum cell sizes”,⁠² which shows that minimum-cell rules and perturbation are alternatives, chosen together with a disclosure review board.

Note the remedy: NCES prefers recategorization to blanks. Merging categories keeps the total information visible at a coarser level instead of creating holes that invite subtraction.

k-anonymity and its known failures

Latanya Sweeney’s 2002 paper formalized k-anonymity: “a release provides k-anonymity protection if the information for each person contained in the release cannot be distinguished from at least k-1 individuals whose information also appears in the release”.⁠⁴ Sweeney also examined “re-identification attacks that can be realized on releases that adhere to k-anonymity unless accompanying policies are respected”.⁠⁴

Machanavajjhala and colleagues then showed “using two simple attacks that a k-anonymized dataset has some subtle but severe privacy problems”: an attacker “can discover the values of sensitive attributes when there is little diversity in those sensitive attributes,” and “k-anonymity does not guarantee privacy against attackers using background knowledge”.⁠⁵ They proposed ℓ-diversity, which requires variety in the sensitive attribute within each group.

The homogeneity problem maps directly onto priority tables. If a published cell says that all 7 participants from a small precinct who are over 75 ranked “health and care” first, the count of 7 passes a k = 5 rule, and yet anyone who knows a participant in that group learns their answer exactly.

Subtraction defeats isolated suppression

NIST’s guidelines summarize the deeper problem: “even aggregate statistics can inadvertently leak information from individual data points,” and publishing enough statistics allows “accurate reconstruction attacks”.⁠⁶ Suppression rules that look at one cell at a time are exposed in three ways. The following are worked illustrations, labeled hypothetical:

Hypothetical example, within a table. A district publishes a total of 412 participants and counts for nine of ten subjects ranked first, summing to 409. The suppressed cell is 3. Primary suppression without complementary suppression protected nothing.

Hypothetical example, across tables. One table shows first choices by district; another shows first choices by age band for the whole city. If one district contains almost all participants over 75, the second table nearly reveals that district’s older participants’ answers, even if the first table suppressed them.

Hypothetical example, across releases. In March a precinct shows 48 participants, 20 choosing “housing.” In April it shows 49 participants, 21 choosing “housing.” The one new participant chose housing. Because the framework lets people join and revise (see Participation over time), consecutive releases are a differencing channel unless releases are spaced, thresholded on change, or noised.

The cost of silence

NIST observes that protection methods hit small groups hardest and that this “is an unavoidable property of any robust anonymization mechanism”.⁠⁶ Suppression makes the cost visible as blanks. NCHS acknowledges the tension by allowing a proportion that fails its standards to be “identified as unreliable but not be suppressed if it can be interpreted appropriately in the context of subject-specific factors and report objectives”.⁠¹ The standards leave that judgment to a clearance official, which means a person, accountable for the decision.

Competing interpretations

“Suppress generously; the harm of exposure outweighs the harm of a blank.” On political priorities, exposure can invite pressure or retaliation, which is the concern behind Intimidation and Can someone be punished for an opinion?. A blank harms no one directly.

“Suppression silences small communities.” The NIST observation,⁠⁶ cuts the other way: rural precincts, small language communities and minority groups are exactly the cells that disappear. A platform meant to surface overlooked priorities defeats itself if its rules erase them.

“Coarsen, do not delete.” NCES’s preference for recategorization⁠² supports merging small areas or categories so that small communities appear, with less detail, rather than not at all.

“Rules are a false comfort; use formal privacy.” Because subtraction and reconstruction defeat cell rules⁠⁶ some argue for noise-based methods with provable guarantees. Others respond that noise is harder for the public to read than a blank, and that a clear rule people understand has its own legitimacy value.

Application to this framework

These are design proposals for My Vote Forward, not results. The minimum group size remains a provisional choice not yet set, as stated in Privacy and Measurement.

1. Run two tests on every cell. A disclosure test (a minimum count and a diversity check so that no published group is unanimous on a subject when it is small) and a reliability test (an interval width rule modeled on NCHS,⁠¹ adapted for participant data rather than survey estimates).
2. Apply complementary suppression automatically. No release publishes a total together with all but one of its parts. The release software checks row, column and grand totals before publication.
3. Prefer coarsening. Following NCES,⁠² merge adjacent precincts or age bands before blanking cells, and say in the release which categories were merged.
4. Control differencing over time. Publish on a fixed schedule rather than continuously, require a minimum number of new participants in a cell between releases before updating it, and document the rule.
5. Report what was withheld. Each release states how many cells were suppressed or merged, and for which reason, so readers know what is missing and why. This mirrors the missingness reporting proposed in Measurement.
6. Name the accountable person. As NCHS assigns review to a clearance official,⁠¹ each release names who approved exceptions, which ties to Published releases.

Unresolved questions

• What minimum count and what diversity rule should apply to political priority data, which may be more sensitive than health or education statistics?
• Should reliability rules from survey statistics apply to participant data at all, given that participant data has no sampling design? An interval for self-selected participants describes a different kind of uncertainty.
• How long should the interval between releases be to prevent differencing without making results stale?
• Should a small community be able to request that its figures be published in coarser form rather than suppressed, and who could speak for it?
• When suppression and differential privacy are combined, which should be applied first, and how should their joint effect be explained?