Colorado’s risk-limiting audits: statistical evidence from paper ballots

Colorado’s risk-limiting audits (RLAs) are post-election checks in which bipartisan county audit boards examine randomly selected paper ballot cards, chosen from a seed generated by public dice rolls, until there is strong statistical evidence that a full hand count would confirm the reported winners.⁠^2,3 Colorado’s legislature required RLAs in 2009, and the state ran the first statewide RLA in the November 2017 coordinated election; current rules cap the risk limit for statewide contests in comparison audits at 5 percent, and the Secretary of State set 3 percent for 2026.⁠^2,3,4

In plain words

After an election, how do you know the counting machines got it right? You could count every paper ballot by hand, but that is slow and expensive, and people make mistakes too. Colorado does something smarter.

It pulls a random sample of real paper ballots and has two people from different parties look at each one and record what the voter marked. Then a computer compares what they saw with what the scanner said about that same ballot. If they match well enough, the state can be very confident the winners are right. If not, it checks more ballots, all the way up to a full hand count if needed.

The random choice of ballots comes from rolling dice in a public meeting, so nobody can secretly pick which ballots get checked. Close races need more ballots checked; landslides need only a few.

Defined terms

• Risk-limiting audit (RLA). Colorado’s rule defines it as a post-election audit of paper ballot cards “which has a pre-specified minimum chance of requiring a full hand count if the outcome of a full hand count would differ from the reported tabulation outcome”.⁠³
• Risk limit. “The largest statistical probability that an incorrect reported tabulation outcome is not detected and corrected”.⁠³ It is not the chance the result is wrong; Lindeman and Stark stress that an RLA “cannot harm correct outcomes” and that, with a 10 percent risk limit, there is at least a 90 percent chance it corrects a wrong outcome.⁠⁵
• Comparison audit. Auditors compare voter markings on sampled ballot cards with the voting system’s cast vote record (CVR) for each card.⁠³
• Ballot polling audit. Auditors tally markings on sampled cards without comparing them to individual CVRs; used where systems cannot export CVRs.⁠³
• Ballot manifest. The county’s inventory that uniquely identifies where every tabulated ballot card is stored, so a random selection can be retrieved.⁠³
• Diluted margin. The smallest reported margin in votes divided by the number of ballots cast.⁠⁵
• Software independence. The property that an undetected software error cannot cause an undetectable change in the outcome; RLAs “leverage” it by checking a well kept paper trail.⁠⁵

For the general concept and how it relates to ballot secrecy, see audits and verification.

What happened

Colorado’s path to statewide RLAs took eight years. Before the change, Colorado audited a fixed 5 percent of scanners and direct recording devices; the Secretary of State’s office later wrote that “there is no mechanism by which the audit can correct an incorrect outcome” under that model.⁠⁶ In 2009 the General Assembly enacted section 1-7-515, C.R.S., requiring RLAs,⁠² originally by the 2014 general election; a 2013 law moved the deadline to the 2017 coordinated election.⁠⁶

A federally funded pilot ran into practical obstacles. Counties raised concerns about cost, time, and whether ballots would be subject to the state open records law without protecting voter anonymity, which stalled the project until legislation resolved it.⁠⁶ Arapahoe County conducted the state’s first RLA pilot in November 2013.⁠⁶

In November 2017, Colorado held a public seed ceremony, using rolls of ten-sided dice, and began what NPR reported as a first-in-the-nation statewide risk-limiting audit; Secretary of State Wayne Williams said, “This is in many ways a new day for ensuring the integrity of elections”.⁠⁷ The 2017 audits used open source software, RLATool, built by the firm Free & Fair;⁠^8,7 the state now describes its audit software as an open source application created by the Colorado Department of State.⁠² Statisticians who advised the program noted that the 2017 audits covered only contests contained within a single county, and that auditing contests crossing county lines required new methods.⁠⁸

The program has run after every election since. After the November 2020 general election, in which 3,291,548 Coloradans voted, the Secretary of State announced that the statewide audit “provides a high statistical confidence in the election results,” and that each county’s bipartisan canvass board certified after the audit.⁠^9,10

How it worked

Colorado’s procedure, as set out in Election Rule 25 (version effective March 2, 2026), runs in fixed steps:⁠³

1. Risk limits published in advance. No later than 32 days before election day, the Secretary of State publishes the risk limits. In comparison audits, the limit “will not exceed five percent for statewide contests, and ten percent for countywide contests.”
2. Manifests and CVRs committed. Counties hash and upload their ballot manifests and CVR files, so the data cannot be changed after the sample is drawn without detection.
3. Target contests chosen. By 5:00 p.m. the Friday after election day, the Secretary of State selects at least one statewide contest and at least one other contest per county, considering closeness, scope, any “cause for concern,” feasibility, and recommendations from clerks or the public. The target contest with the closest diluted margin sets the sample size.
4. Public random seed. On the thirteenth day after the election, at a publicly noticed meeting, members of the public chosen at random take turns rolling a 10-sided die to produce a seed of at least 20 digits, which feeds a SHA-256 based pseudo-random number generator to select specific ballot cards.
5. Bipartisan examination. Audit boards retrieve each selected card (the original if it was duplicated), interpret it using the state’s voter intent guide, and record the markings; if members cannot agree, they say so.
6. Comparison and escalation. The state compares the reports to the CVRs and posts the results. “The RLA will continue until the risk limit for the target contests is met or until a full hand count results”; the Secretary of State may order more rounds, a full hand count, or delay canvass.
7. Privacy cleanup. Counties must redact CVR entries that could identify an individual voter.

The statistics determine how many ballots are needed. Lindeman and Stark give a simple rule for a comparison audit at a 10 percent risk limit: the audit can stop after n ballots when n is at least (4.8 + 1.4(o1 + 5o2 − 0.6u1 − 4.4u2)) divided by the diluted margin, where o and u count one vote and two vote overstatements and understatements.⁠⁵ With a 5 percent diluted margin and no discrepancies, 96 ballots suffice; each one vote overstatement adds 1.4 divided by the margin.⁠⁵ Overstatements (errors that favored the reported winner) raise the sample size; understatements lower it, but less.⁠⁵ Colorado’s FAQ states the same principle: “The closer the margin, the more ballots are audited”.⁠²

Evidence: measured findings

Colorado’s public record on its RLAs includes the following figures, each from the source cited:

• Pilot scale (2013). In Arapahoe County’s pilot, “based on the diluted margin ... 44 ballots were selected for the audit,” using a seed from rolls of ten sided dice.⁠⁶ One selected ballot “could not be verified against its CVR,” because finding it in a large batch would have required rescanning the batch.⁠⁶
• Coverage. The statisticians advising the state estimated that by June 2018, 98.2 percent of active Colorado voters would be in counties whose systems export CVRs, allowing the more efficient comparison audits.⁠⁸ Counties without CVR export conduct ballot polling audits.⁠³
• Risk limit over time. The state’s FAQ says Colorado started with a 9 percent risk limit and reduced it to 3 percent;⁠² the Audit Center lists 3 percent for the 2026 general election.⁠⁴
• Outcomes. The Secretary of State reported that the 2020 statewide RLA was completed and provided “high statistical confidence” before certification.⁠⁹

This review could not open Colorado’s per-election discrepancy reports (links found in searches returned “not found” on October 26, 2026); the state’s Audit Center now directs requests for historical audit data to the elections division by email.⁠⁴ Totals of ballots examined and discrepancies by year are therefore not stated here.

Failures and criticisms

Colorado’s own pilot documented practical failure modes. Staff reported that parallel scanning was “significantly more time-consuming” and required ballots to stay in scan order, which “allows significant opportunity for human error”; they observed human error in both hand counting (miscounting stacks) and scanning (not storing ballots in scan order).⁠⁶ They recommended batch sizes of 50 ballots or fewer for retrieval.⁠⁶

RLAs rest on assumptions that must themselves be checked. Lindeman, McBurnett, Ottoboni, and Stark write that RLAs “require a durable, voter-verifiable record of voter intent” and assume the audit trail is complete and accurate, an assumption “not automatically satisfied: a compliance audit ... is required”.⁠⁸ Lindeman and Stark warn that an inaccurate ballot manifest can invalidate the risk limit.⁠⁵ See the glossary for chain of custody.

The method audits tabulation, not everything. An RLA checks whether the paper supports the reported winners. It does not check voter eligibility, signature verification, ballots never returned, or whether a ballot marking device printed what the voter intended. Stark and Xie argue that common pre-election and parallel tests “cannot reliably detect outcome-altering problems” in ballot marking devices;⁠¹¹ where voters do not carefully check machine printed ballots, the paper trail itself may not reflect intent. In Colorado’s 2020 general election, 3,092,903 of 3,291,548 ballots were mail ballots;⁠¹⁰ because mail ballots are generally marked by hand (this site’s inference, not a figure the state reports), the concern applies mainly to in-person voting there.

Scope was a criticism early on: the 2017 audits could not yet audit cross-county contests efficiently.⁠⁸ And because the state targets a limited set of contests, most local contests in a given election are not audited to a risk limit.

Competing interpretations

RLAs as the gold standard. Supporters, including the National Academies, recommend that states “mandate a specific type of audit known as a ‘risk-limiting’ audit” before certification.⁠¹ On this view, Colorado shows that statistically rigorous, public, bipartisan audits are feasible at statewide scale, and that public dice rolls and published manifests make the process inspectable by anyone.

RLAs as necessary but not sufficient. Others accept RLAs but stress what they omit: eligibility, custody, and the integrity of the paper itself.⁠^8,11 Public confidence surveys, eligibility audits, and signature verification audits answer different questions.

Skeptics of statistical assurance. Some members of the public distrust any audit that examines a sample rather than every ballot, and some prefer full hand counts. Defenders respond that hand counts also produce errors, as Colorado’s pilot observed,⁠⁶ and that a sample chosen by public dice rolls with a stated risk limit gives quantified evidence. This disagreement is partly about what kind of evidence persuades people, not only about statistics; see audits and verification.

Lessons

These are interpretations drawn from the record.

1. Evidence can be cheaper than certainty. Effort scales with margin: wide margins need few ballots, close ones many.⁠⁵
2. Logistics decide feasibility. Ballot accounting, batch size, and retrieval mattered as much as statistics in the pilot.⁠⁶
3. Publicity is part of the method. Pre-published risk limits, hashed manifests, and public dice let observers check that nobody chose the sample.⁠³
4. Privacy must be designed in. Open records concerns about ballot anonymity stalled the pilot until law addressed them, and the rule now requires redacting identifiable CVR entries.⁠^6,3
5. Legal mandates need years and iteration. From the 2009 statute to the 2017 first statewide audit took eight years.⁠^2,6

Application to this framework

Colorado’s RLAs model the kind of assurance the My Vote Forward framework asks of any official voting component: evidence about ballots, not voters, that does not depend on trusting software (see audits and verification). The framework’s “My vote” principles require that verification and secrecy coexist; Colorado’s rule shows one concrete way, by auditing anonymous ballot cards and redacting identifiable records.

The case also informs how the framework treats its non-binding parts. Public collective priorities and voluntary opinions are not ballots, but published releases about them should still follow fixed methods announced in advance, published inputs, and randomness that observers can verify, the same habits that make Colorado’s audits credible (see published releases).

Unresolved questions

• What do Colorado’s year by year audit records show about discrepancy rates and their causes? This review could not open the per-election reports.
• How should RLAs extend efficiently to small local contests that appear on few ballots?
• What compliance checks on custody and manifests should be published so the public can assess whether the paper trail is trustworthy?
• Do public audits measurably change voter confidence, and among whom?
• How should RLAs apply where most in-person voters use ballot marking devices rather than hand marked ballots?