Why should anyone trust a published release?

No one should trust a My Vote Forward release because the operator says it is accurate. A release earns credibility only by publishing what a skeptical reader needs to check it: who took part, how they were verified, the exact wording and list version, the counting method, what was excluded and why, and the limitations. That standard is borrowed from existing practice, including the American Association for Public Opinion Research’s disclosure code and the logic of risk-limiting audits in elections. The framework has published no real release yet, has no independent auditor, and must withhold some detail to protect privacy, so the honest answer today is that trust is not yet earned.

The objection in its strongest form

The objection to My Vote Forward here is that a published release is an assertion by an interested party. The operator writes the questions, runs the system, counts the results, and decides what to publish. Even with good intentions, it could make errors no one detects. With bad intentions, or under pressure from funders or partners, it could adjust methods after seeing results, drop inconvenient data, or frame findings to suit a narrative. Publishing a methods document does not help, the critic says, because almost no one reads it, and those who do cannot check the underlying data, which is withheld for privacy.

A deeper version notes that trust in institutions is low and polarized. Gallup’s 2025 survey found 28% of Americans trust mass media a great deal or a fair amount.⁠⁶ A new organization publishing numbers about public priorities enters that environment with no track record. Readers who dislike a result will assume it was rigged; readers who like it will not check.

Why the objection matters

Release credibility matters for My Vote Forward because releases are the only way participant data leaves the system, and the only product institutions and the public see. If releases are not credible, nothing else the framework does matters. The objection also matters because the framework’s privacy protections, which withhold individual data, make independent checking harder. See published releases for the release design and why trust this operator? for the institutional question.

Defined terms

• Published release: a scheduled package of group results with stated methods, denominators, and consent basis. See published release.
• Disclosure standard: a professional rule specifying what must be reported with a result.
• Pre-registration: publishing an analysis plan before seeing results, so methods cannot be chosen to fit them.
• Risk-limiting audit: an election audit that has a large, pre-specified chance of correcting a wrong outcome. See risk-limiting audit.
• Software independence: a property in which an undetected change in software cannot cause an undetectable change in an outcome.

Evidence

Survey research has a mature disclosure standard. AAPOR’s Code, revised in April 2021, requires immediate disclosure with any public report of items including the data collection strategy; who sponsored and conducted the research, and the original funding source if different; the exact wording and presentation of measurement tools; the population studied; how the sample was generated and recruited, including whether it was probability-based; methods and modes of collection; dates; sample sizes; how data were weighted; how data were processed and checked for quality; and a general statement acknowledging the limitations of the design and data collection.⁠¹

Election auditing offers a model for trust based on evidence rather than authority. Mark Lindeman and Philip Stark describe a risk-limiting audit as “a method to ensure that at the end of the canvass, the hardware, software, and procedures used to tally votes found the real winners”; such audits “do not guarantee that the electoral outcome is right, but they have a large chance of correcting the outcome if it is wrong”.⁠² The risk limit is “the largest chance that the audit will stop short of a full hand tally when the original outcome is in fact wrong, no matter why it is wrong”.⁠² Colorado’s statute requiring risk-limiting audits dates to 2009, its first statewide audit took place in the 2017 coordinated election, and the random seed that selects ballots for audit is “generated by rolling 20 ten-sided dice during a public meeting”.⁠³ The public dice roll is a small detail with a large point: a step that could be manipulated is done where anyone can watch. See the Colorado case.

The limits of published numbers are also documented. Pew Research Center found that an online opt-in poll showed 20% of adults under 30 agreeing the Holocaust is a myth, compared with 3% in Pew’s probability-based panel, and attributed the gap to bogus respondents.⁠⁴ A release can follow every disclosure rule and still be misleading if the sample is poor; disclosure lets readers see the problem, not fix it.

Privacy limits how much can be published. NIST’s guidance on differential privacy warns that “even aggregate statistics can inadvertently leak information from individual data points” and that “publishing enough statistics will always result in accurate reconstruction attacks”.⁠⁵ A release that published every cross-tabulation to allow checking would expose participants. Trust and privacy must be balanced, not maximized separately.

How comparable systems have handled it

The comparable systems use four mechanisms.

• Disclosure. Professional survey codes require methods and limitations with every result.⁠¹
• Public procedure. Colorado’s public dice roll and audit process let observers see key steps.⁠³
• Independent checking. Audits are designed so that the people who counted are not the only ones who verify.⁠²
• Track record. Long-running pollsters and statistical agencies earn credibility partly by being right, and visibly wrong, over many releases.

Competing interpretations

One interpretation holds that disclosure is sufficient: if methods are published and follow professional standards, readers have what they need, and further verification is the job of critics and researchers.

A second interpretation holds that disclosure without independent verification is not enough for politically consequential numbers. On this view, releases need an outside auditor with access to underlying data under confidentiality rules, as election audits have outside observers.

A third interpretation holds that credibility comes mainly from track record and institutional independence, and no procedure can substitute for years of reliable releases by an organization with no stake in the results.

What the framework answers

My Vote Forward’s release rules, which apply to any real release:

• AAPOR-style disclosure of sponsor, funder, wording, list version, population, recruitment, verification level, dates, sizes, weighting, processing, exclusions, and limitations.⁠¹ See published releases.
• Pre-registered methods. The aggregation and exclusion rules are published before participation opens, so they cannot be chosen after seeing results. See evaluation.
• Results with and without exclusions and weights, so readers see how much each step matters.
• Version and correction log. Every release has a version; corrections are dated in the changelog rather than silently replaced.
• Privacy review before publication, including suppression of small groups. See privacy.
• Public steps where possible. Random selections used in checking, such as samples drawn for verification, should use seeds generated in public, following the Colorado model.⁠³

Hypothetical example. Before a pilot opens, the operator publishes its counting method and its rule for excluding accounts created during a burst of identical submissions. After the window closes, the release shows results under the pre-registered method, the number excluded, and results without exclusions. A critic who suspects the exclusion rule favored one subject can compare the two sets directly.

Where the objection still wins

The objection wins today on three points. First, there is no real release and no track record; every safeguard above is a plan. Second, there is no independent auditor; the operator would be checking itself. Third, privacy prevents full public verification: readers cannot re-run the count on individual data, and must rely on whoever has access. Disclosure also cannot fix a weak sample, as Pew’s opt-in findings show.⁠⁴ A reader who distrusts the operator has good reason to withhold trust until independent verification exists.

Unresolved questions

• Who should audit releases, with what access to underlying data, and who pays them without compromising independence?
• What is the right balance between detailed breakdowns that allow checking and privacy protections that limit them?
• Can cryptographic or differential privacy methods let outsiders verify counts without seeing individual data?
• How many releases would it take to establish a track record, and how would errors be scored?

Related

• Published releases
• Audits and verification
• Why trust this operator?
• Risk-limiting audit
• Hard questions