Collective priorities: how individual rankings become a group picture, and what that picture cannot say

Plain-language summary

If you ask one person to rank what matters most, you get a clear list. If you ask thousands of people, you have to decide how to combine their lists. That step sounds simple. It is not.

Different ways of combining the same answers can produce different winners. Under some conditions, a group's combined preferences can even go in a circle: most people prefer A to B, most prefer B to C, and most prefer C to A. Mathematicians have proved that no method can meet every reasonable fairness test at once.

The form people fill out also matters. Asking people to rank items, to approve some of them, or to split a fixed budget can lead to different results from the same people.

And a combined list only describes the people who took part, on the question they were asked, at the time they answered. It is not an order to any government. This page explains the methods, their known problems, and how My Vote Forward treats a published list of shared priorities.

Defined terms

• Collective priority: a published summary of how a group of participants, taken together, rated the relative importance of a set of subjects. It is a description of participants, not a decision. See the glossary.
• Aggregation rule: the procedure that turns many individual answers into one group result.
• Ranking: ordering items from most to least important. A ranking says which item is higher, not by how much.
• Allocation: dividing a fixed amount (money, points or tokens) among items. Allocation forces a tradeoff; see tradeoffs.
• Approval ballot: the voter marks every item they support, with no order.
• Condorcet cycle: a situation where majority preferences between pairs of options form a loop.
• Strategic voting: reporting something other than one's true preference to improve the outcome for oneself.
• Coordinated participation: organized efforts to submit many responses toward a goal. It ranges from legitimate mobilization to fraud using fake or stolen identities.
• Category coverage: whether the list of subjects offered lets participants express what actually matters to them.

Evidence

Research result: majority preferences can be inconsistent

The Stanford Encyclopedia of Philosophy entry on social choice theory, by Christian List, presents Condorcet's paradox with three equal groups of voters: there are majorities "for x against y, for y against z, and for z against x"¹. The general lesson it draws is that "majority preferences can be 'irrational' (specifically, intransitive) even when individual preferences are 'rational'"¹.

Research result: no rule satisfies every fairness condition

The same entry states Arrow's theorem: no rule for combining individual rankings into a group ranking can satisfy all of a short list of conditions at once¹. The conditions are universal domain (the rule must handle any combination of individual rankings), weak Pareto (if everyone prefers x to y, so does the group), independence of irrelevant alternatives (the group's ranking of x and y depends only on how individuals rank x and y), and non-dictatorship (no single person always determines the result)¹.

For decisions between exactly two options, the picture is cleaner. May's theorem, as stated in the entry, shows that a rule satisfying universal domain, anonymity, neutrality and positive responsiveness is exactly majority rule¹. The difficulties begin with three or more options, which is the normal case for a priority list.

Research result: popular methods trade one problem for another

• The Borda count, which scores each item by its position on each ranking, "avoids Condorcet's paradox but violates one of Arrow's conditions, the independence of irrelevant alternatives," and it is not strategy-proof¹. In plain terms, adding or removing an unrelated item can change the order of the others, and participants can gain by ranking insincerely.
• Approval voting lets each voter support any number of options; it is strategy-proof when each voter's preferences simply divide options into acceptable and unacceptable, and loses that property otherwise¹.
• The Gibbard and Satterthwaite theorem, as summarized in the entry, shows that under general conditions every non-dictatorial rule with three or more possible outcomes gives some voter an incentive to vote strategically¹.

Research result: ballot format changes outcomes

Participatory budgeting offers direct evidence that the form of the question matters. Goel and colleagues, in work first published in ACM Transactions on Economics and Computation in 2019, examined the voting method commonly used in participatory budgeting, identified drawbacks, and proposed an alternative they call knapsack voting². A later study by Gelauff and Goel, using paired ballots from the Stanford Participatory Budgeting platform, found that "in most elections, voters selected significantly more expensive projects using K-approval than using knapsack," that the effect was shaped by both the voting method and "the explicit constraints of the voting interface," and that more complex ballots took longer but were "not correlated with a higher abandonment rate"³.

Quadratic voting is another proposal for capturing how strongly people care. Lalley and Weyl describe a design in which individuals buy votes with "voice credits" at a quadratic cost, and argue it can address the failure of existing democracies "to incorporate intensity of preference"⁴. They call it "still-nascent"⁴. Its use for public decisions is a research question, not settled practice.

Documented history: coordinated and fraudulent participation

Open participation can be flooded. A 2021 report by the New York State Office of the Attorney General found that fake comments made up nearly 18 million of the more than 22 million comments the Federal Communications Commission received in its 2017 net neutrality rulemaking⁵. The report found that a campaign funded by broadband companies generated more than 8.5 million fake comments supporting repeal, and separately that a 19-year-old college student submitted over 7.7 million fake comments supporting net neutrality using fictitious identities⁵. Fraud appeared on both sides of the question⁵. The report also states that it had not found evidence that the funding companies had direct knowledge of the fraud⁵.

Documented design: looking for agreement rather than only counting

vTaiwan's use of Pol.is, as described in a 2018 report, does not allow replies to comments; participants vote on statements, and the system groups people who vote similarly, which lets organizers look for statements that win support across groups⁶. This is an example of an aggregation design that reports structure (where groups agree and disagree) rather than one ranked list. See cases for its limits.

Competing interpretations

• "Use the simplest rule and accept its flaws." Plurality or average rank is easy to explain, and explainability supports trust. Critics answer that simple rules hide the paradoxes rather than avoid them¹.
• "Use the rule with the best formal properties." Some prefer methods that respect majority comparisons between pairs. Others note that every rule fails some condition¹, so "best" depends on which failure you accept.
• "Measure intensity, not only order." Allocation and quadratic methods try to capture how much people care²⁴. Critics worry about complexity, about whether people understand the cost structure, and about uneven ability to use them.
• "Report the distribution, not a winner." On this view a priority release should show how answers were spread, where groups differ, and how stable the order is under different rules, rather than one headline list. The cost is that results become harder to summarize.
• "Open participation is democratic; controls are gatekeeping." Against this, the FCC case shows that open intake without identity checks can be dominated by fabricated volume⁵. The opposite risk is that heavy verification excludes real people; see access and exclusion.

Application to this framework

This section is design proposal and interpretation, not a measured result.

• Publish the method with the result. Any collective priority release should name the aggregation rule, the ballot format, the subject list and its version, and show whether the order changes under at least one alternative rule. Because rules disagree¹, this is the minimum needed for a reader to judge the result. See published releases.
• Keep ranking and allocation distinct. A ranking of importance and an allocation of a limited budget are different questions and can produce different results³. The framework treats them separately; see priorities and tradeoffs.
• Count confirmed selections, not defaults. An untouched default is not a preference. Releases should report how many participants actively set each answer. See measurement.
• State coverage limits. A fixed list of subjects shapes what can rank high. The question of who sets that list is an agenda power question addressed in governance. Hypothetical example: if a list of ten subjects omits housing, a participant for whom housing matters most has no accurate way to say so, and the published order will understate it.
• Expect coordination and design for it. Mobilizing people to participate is legitimate. Fabricated or duplicated identities are not⁵. The framework distinguishes one-person-one-response controls from verification of electoral eligibility; see identity and registration.
• No automatic mandate. A collective priority list describes participants. It does not tell an institution what to do, does not override legal commitments or rights, and is not a ballot. Advisory processes elsewhere have been ignored by governments⁶, and binding processes are still bounded by law; see cases, proposals and representation.
• Priorities are public; ballots are not. Collective priorities can be published as aggregates. Private votes are a different data class and are never part of this aggregation; see privacy and opinion and choice.

Unresolved questions

• Which aggregation rule should be the default for a public priority list, and should the default change with the number of subjects?
• How large must a group be, and how stable must an order be under alternative rules, before a ranked position is worth publishing?
• How should a release show that two subjects are effectively tied rather than presenting a false order?
• What identity assurance is proportionate for a nonbinding priority exercise, given both the FCC experience⁵ and the risk of excluding real participants?
• Can intensity measures such as allocation or quadratic methods⁴ be made understandable to a general public, and how would that be tested? See evaluation.