Synthetic data: artificial records made to resemble real data

Synthetic data is artificial data generated to resemble real data or a defined scenario. NIST describes synthetic data generation as “a process in which seed data are used to create artificial data that have some of the statistical characteristics of the seed data.” The My Vote Forward demo uses synthetic data throughout, labeled “Simulated data,” and never presents it as real participation.

Definition

NIST’s glossary gives two closely related definitions, from SP 800-188 and NISTIR 8053: synthetic data generation is a process in which seed data are used to create artificial data that have some of the statistical characteristics of the seed data.⁠¹ NIST SP 800-226 describes a synthetic dataset as one that “looks like the original dataset in that it has the same schema and attempts to maintain the properties of the original dataset (e.g., correlations between attributes),” but that “consists of algorithmically generated data associated with ‘fake’ individuals”.⁠²

Two kinds should be distinguished:

• Data derived from real records, where a model learns patterns from real people’s data and generates new records.
• Data from a defined scenario, where records are generated from assumptions with no real people behind them. The My Vote Forward demo uses this kind.

Origin and history

Synthetic data generation entered government privacy guidance as one way to share useful data without releasing real records; the NIST definition above comes from SP 800-188, NIST’s guidance on de-identifying government datasets.⁠¹ Privacy research then showed that synthetic data is not automatically safe. NIST SP 800-226 states that synthetic data techniques that do not satisfy differential privacy “generally provide only informal privacy guarantees,” that “there are many reports of privacy attacks against non-differentially private synthetic data that have successfully revealed the original data,” and lists as a privacy hazard that “synthetic data generated without differential privacy may be susceptible to privacy attacks”.⁠²

How synthetic data is used and judged

Synthetic data serves three main purposes:

• Demonstration. Showing how an interface or report would look without exposing anyone.
• Testing. Exercising software, including edge cases such as tiny groups or extreme values, before real data exists.
• Research access. Letting analysts explore a dataset’s structure before applying for access to the real one.

It is judged on two axes. Utility asks whether analyses on the synthetic data give answers close to those on real data. Disclosure risk asks whether anything about a real person can be learned from it. NIST notes that because synthetic data “looks like the original data,” it is “particularly easy to use”;⁠² that ease is also why it can be mistaken for real evidence.

Common confusions and neighboring terms

• Synthetic vs. de-identified data. De-identified data is real records with identifiers removed. Synthetic records are generated. Both can leak information if not built with formal protections.
• Synthetic vs. simulated. On this site the terms are used together: the demo’s data is synthetic, generated from a simulation of a fictional community.
• Synthetic data vs. aggregate. An aggregate summarizes real answers; synthetic data invents records.
• Synthetic data vs. differential privacy. Differential privacy is a guarantee; synthetic data is a format. Differentially private synthetic data combines them.
• Synthetic data vs. fabricated evidence. Synthetic data presented as real participation would be fabrication. The label is what separates them.

How My Vote Forward uses the term

My Vote Forward’s demo uses scenario-based synthetic data for every community priority, schedule, and sample ballot, labeled “Simulated data.” Its demo release packages contain only clearly labeled synthetic data, and current demo users cannot opt into a real program, because none exists. See Published releases, Status, and About. Because the demo’s data is not derived from real records, it carries no disclosure risk for real people; it also carries no evidence about real people.

Unresolved questions

• If a real program ever existed, should its public test datasets be generated with differential privacy, accepting lower utility?
• How should screenshots of the demo be labeled so that “Simulated data” survives when they are shared elsewhere?
• What features of a synthetic community make a demonstration realistic without implying predictions about a real one?