Differential privacy in practice: budgets, sensitivity, noise, and who bears the error

Differential privacy is a mathematical standard for publishing statistics so that the output barely depends on whether any one person’s data was included. In practice it means adding random noise calibrated to how much one person could change a statistic, and accounting for total privacy loss across every release as a “privacy budget.” NIST’s 2025 guidelines describe both its strength, a guarantee that holds against outside data, and its cost, error that falls hardest on small groups.

In plain words

Suppose a town publishes how many participants put “parks” in their top three. If you knew everyone except your neighbor had answered, and you saw the count go up by one after your neighbor joined, you would learn your neighbor’s answer. Removing names does not stop this.

Differential privacy stops it by adding a small random amount to each published number, so that one person’s answer is hidden inside the randomness. The trade is accuracy. The random amount is about the same size whether a group has 10 people or 10,000. For 10,000 people a few units of noise hardly matter. For 10 people they can swamp the real number.

There is also a running total. Each time you publish from the same data, you spend some privacy. Publish too often, or in too much detail, and the protection wears thin.

The glossary entry defines the term and its history. This page works through how it would apply to priority data, with the arithmetic. Simple minimum-group rules are covered in Small-group suppression.

Defined terms

• Neighboring datasets. Two datasets that “differ in the data of one individual”.⁠¹
• Pure ε-differential privacy. A randomized mechanism M satisfies it if, for all neighboring datasets D1 and D2 and every set of outcomes S, Pr[M(D1) in S] <= e^epsilon * Pr[M(D2) in S].⁠¹
• (ε, δ)-differential privacy. A relaxation that allows the bound to fail with small probability δ.
• Unit of privacy. What counts as one individual’s contribution: one answer, one day of answers, or everything a person ever submitted.⁠¹
• Sensitivity (L1). The largest change in a query’s output, summed across all output numbers, when one individual’s data is added or removed.⁠¹
• Privacy budget. The total ε allowed across all analyses of the same data.⁠¹
• Post-processing invariance. The property that any computation on a differentially private output, done without the raw data, remains differentially private.⁠¹
• Invariant. A statistic published exactly, outside the privacy accounting, as the Census Bureau did with some totals.⁠²

Evidence

What the guarantee says, and what it does not

NIST Special Publication 800-226, Guidelines for Evaluating Differential Privacy Guarantees (March 2025), gives the definition above and explains ε as “the privacy parameter, the privacy loss or the privacy budget”.⁠¹ The guarantee is a ratio bound: whatever an observer concludes from the output, they would have concluded nearly the same thing, within a factor of e^ε, had any one person’s data been absent. For ε = 1 that factor is about 2.72; for ε = 0.1 it is about 1.105.

NIST explains why aggregation alone does not suffice: “even aggregate statistics can inadvertently leak information from individual data points,” and “there are mathematical results that show that publishing enough statistics will always result in accurate reconstruction attacks,” with “practical examples of such attacks being performed successfully on real statistical releases”.⁠¹ A civic platform that publishes many cross-tabulations of the same participants is exactly the setting those results describe.

Choosing ε is a policy decision

NIST does not set a universal ε. It states that “how to set ε is still an active area of research,” that one study “suggests that ε of 0.1 generally provides strong privacy protection, and that ε values less than 1 are considered to be reasonable,” and that “many deployments of differential privacy have used larger values (i.e., 1 < ε ≤ 20)”.⁠¹ It warns that “differentially private releases with ε of 10 or greater can leave outliers vulnerable to privacy leakage,” and that for a simple mechanism answering one question “even releases with ε = 1 can help an adversary make more confident guesses”.⁠¹ It also states that “publishing privacy parameters establishes trust and accountability” and “does not create any additional risks to data privacy”.⁠¹

The unit of privacy decides what is protected

NIST gives a concrete warning: “a user-day privacy guarantee with ε = 1 may appear to be a strong guarantee ... but the total privacy loss may be ε = 365 over the course of an entire year”.⁠¹ For a platform where people can revise their priorities at any time (see Participation over time), the natural unit is the person across their whole history, not a single submission. A per-submission guarantee would let someone who updates weekly leak far more than the headline ε suggests.

Sensitivity of priority queries

NIST describes the two standard mechanisms: the Laplace mechanism, which “adds random noise drawn from the Laplace distribution” and “uses L1 sensitivity,” and the Gaussian mechanism, which uses L2 sensitivity and suits high-dimensional outputs.⁠¹ For the Laplace mechanism the noise scale is the L1 sensitivity divided by ε, written Lap(Delta_1 / epsilon).⁠¹

The sensitivity of a priority query depends on its design. The following are worked derivations, not findings:

• First-choice histogram over 10 subjects. Each person contributes one count to one bin. Removing a person changes one bin by 1. Δ1 = 1.
• Top-three histogram. Each person adds 1 to three bins. Δ1 = 3.
• Full-rank Borda totals. With scores 9, 8, ..., 0 across ten subjects, one person adds a total of 45 across bins. Δ1 = 45.
• Each person’s whole history, with up to R revisions published as separate snapshots. Sensitivity, or the budget, multiplies by R unless only one snapshot per person enters each release.

The richer the question, the more noise each published number needs for the same ε. That is a reason to publish fewer, simpler statistics rather than every possible cut.

How big is the noise?

For Laplace noise with scale b, the chance that the absolute error exceeds t is e^(-t/b). So a 95% bound on absolute error is t = b * ln(20), about 3.0 times b. NIST illustrates the same idea with a 95% confidence interval for the absolute error of the Laplace mechanism.⁠¹

Hypothetical example. A district publishes top-three counts for ten subjects with ε = 1 per release. Δ1 = 3, so b = 3 and each count’s 95% error bound is about plus or minus 9. In a district with 4,000 participants where 1,200 chose housing, the published 1,200 plus or minus 9 is accurate to under 1 percentage point of participants. In a precinct with 60 participants where 18 chose housing, plus or minus 9 means the published share could plausibly fall anywhere from about 15% to 45%. The noise is identical. Its meaning is not.

Composition and the budget

NIST explains that “the composition property of differential privacy allows us to add up the individual ε parameters for many analyses of the same data,” giving an example in which “10 individual differentially private analyses ... each with a privacy parameter of ε_i = 0.1” yield “a total privacy budget” of ε = 1.⁠¹ A platform that publishes monthly for a year at ε = 1 per month, on the same participants, has spent ε = 12 under basic composition. Tighter accounting methods exist, but the qualitative lesson holds: frequent detailed releases on the same people consume protection.

Post-processing can protect, or quietly add bias

Post-processing invariance means a platform can round, sort or plot noisy counts without spending more budget.⁠¹ But NIST notes two traps. First, forcing noisy counts to be non-negative introduces statistical bias: changing negative counts to zero adds error that “increases as the privacy parameter ε decreases”.⁠¹ Second, post-processing that uses the original data to make releases consistent “does not fall under differential privacy’s post-processing invariance guarantee” and “thus does not satisfy differential privacy”.⁠¹ A release that adjusts noisy district counts so they sum to an exact, unprotected total has to account for that total separately.

NIST also flags a subtle leak for bar charts: “the result of a bar chart query reveals the identities of the bins in addition to the count for each one, and the presence or absence of a bin can reveal information about an individual”.⁠¹ If a platform allows write-in priorities, publishing only the write-ins that appear would reveal that someone wrote them. The bin list must be fixed in advance.

The 2020 Census: the largest public test

The Census Bureau’s TopDown Algorithm, described by Abowd and colleagues in the Harvard Data Science Review, “creates noisy versions of key queries on the data ... using zero-Concentrated Differential Privacy,” treats some statistics as “invariants ... that the Census Bureau has determined, as matter of policy, to exclude from the privacy-loss accounting,” and post-processes the results into a microdata file used to produce the 2020 redistricting data.⁠²

Its effects were contested. Kenny, Kuriwaki, McCartan, Rosenman, Simko and Imai, in Science Advances (2021), studied the Census Bureau’s then-latest disclosure avoidance system and found that it “systematically undercounts the population in mixed-race and mixed-partisan precincts, yielding unpredictable racial and partisan biases,” that it “leads to a likely violation of the ‘One Person, One Vote’ standard as currently interpreted,” and yet that it “does not prevent accurate predictions of an individual’s race and ethnicity”.⁠³ The Bureau’s own technical account presents the system as privacy-loss accounting for the full set of published tables.⁠² The disagreement that followed was less about whether a tradeoff exists than about where to set it and which statistics to hold invariant.

The burden on small groups

NIST lists bias as a hazard: “the noise introduced by differentially private algorithms impacts smaller groups more than larger ones,” and “since marginalized social groups are often smaller ... the noise can magnify or even create biases”.⁠¹ Its illustration uses population by race in one Massachusetts census district, where for the smallest group, with a true count of exactly 1, “the error bar is larger than the population when ε = 1”.⁠¹ NIST adds that this bias “is an unavoidable property of any robust anonymization mechanism, and is not unique to differential privacy”.⁠¹

Competing interpretations

“Differential privacy is the only defensible standard.” Proponents point to reconstruction results:⁠¹ any platform that publishes many statistics about the same people needs a guarantee that survives outside data, and ad hoc rules provide none.

“Its costs fall on the people civic data should help most.” Critics point to the Census redistricting findings.⁠³ and NIST’s own small-group warning⁠¹ For a civic platform whose purpose includes making small communities visible, noise that erases them defeats the purpose.

“The real choice is about release design, not the mechanism.” A middle view holds that most harm comes from publishing too many fine-grained statistics. Publishing fewer, coarser releases lets a modest ε protect individuals while keeping meaningful accuracy.

“Participation data is not census data.” Priority answers are voluntary and arguably less sensitive than race or household composition. Others reply that political priorities can be sensitive in exactly the ways that matter for intimidation; see Intimidation.

Application to this framework

These are design proposals for My Vote Forward, not results. The framework has not adopted differential privacy; its demo uses synthetic data, so no real individual is at risk there.

1. Decide the unit of privacy first. The person across their entire participation history, so revisions cannot be used to multiply leakage.⁠¹
2. Publish a release plan with a total budget. State in advance how many releases per year, which tables each contains, the ε for each, and the cumulative total. Publishing ε is recommended by NIST and adds no risk.⁠¹
3. Prefer low-sensitivity questions for public release. First-choice and top-three counts (Δ1 = 1 or 3) rather than full Borda totals (Δ1 = 45) when the release is noisy.
4. Compute error before release, for the smallest published group. Every release states the 95% error bound for each count and declines to publish geographies where that bound exceeds a stated fraction of participants. This is where differential privacy meets Small-group suppression.
5. Fix bins in advance. No write-in categories appear in public tables unless the list is predetermined.⁠¹
6. Disclose post-processing. If counts are clamped to zero or rounded, say so and report the resulting bias direction.⁠¹

Unresolved questions

• What ε should a nonbinding civic priority release use, and should it differ for small districts?
• Should the budget renew each year, and if so, what does that mean for a participant who stays for a decade?
• Is it better to publish noisy small-area figures with honest error bars, or to suppress them? The answer may depend on whether the audience can read error bars correctly.
• How should differentially private releases interact with published exact totals, given that consistency adjustments using raw data fall outside the guarantee⁠¹?
• Could local differential privacy, where noise is added on the participant’s device, work for priorities? NIST notes that “the local model is only useful for queries with a very strong ‘signal’,” which ten-subject rankings in small districts may not have.