Why Prediction Markets Are Accurate, and Where That Breaks Down: Wisdom of Crowds, Brier Scores, and Four Ways Markets Fail

In 1988, a small group of economists at the University of Iowa launched something called the Iowa Electronic Markets, letting students trade contracts on US presidential elections for real money capped at $500 a head. Over the next two decades, the IEM came closer to the final result than the national polls about three-quarters of the time, per the Iowa team's review of a dozen years of election-market data (Berg, Nelson and Rietz). The traders were not pollsters. They were undergraduates, hobbyists, and a scattering of professors, and the prices they set kept landing closer to the final result than the surveys that dominated cable news.

That history is the reason prediction markets get taken seriously at all. It is also why the question of whether they are genuinely accurate, or just occasionally lucky, matters to anyone using them to inform a decision. The short answer is that markets are usually well-calibrated within their limits, often match or beat polls and pundits on political and event-driven questions, and fail in specific, identifiable ways. "Often" is doing real work in that sentence. The largest recent study of the field, covering more than 2,500 political markets in the final five weeks of the 2024 US campaign, found the four major venues ranging from 93% of markets beating chance on PredictIt down to 67% on Polymarket, and concluded that its findings "challenge the view that prediction markets necessarily efficiently and accurately aggregate information about political outcomes" (Prediction Markets? The Accuracy and Efficiency of $2.4 Billion in the 2024 Presidential Election). Accuracy is a property markets have in degrees and in conditions, not a property they have by construction. The longer answer is below.

The mechanism is older than the platforms

The idea is older than the phrase. It traces to a 1907 demonstration by Francis Galton, published in Nature under the title "Vox Populi". At a country fair in Plymouth, 787 people guessed the weight of an ox. The individual guesses were wildly varied. The median guess was 1,207 pounds. The actual weight was 1,198 pounds. No single villager came that close. The phrase "wisdom of crowds" came later: it was popularised by James Surowiecki's 2004 book of that name.

Galton's point was statistical, not mystical. When you aggregate independent estimates from people with different information, the errors tend to cancel and the signal survives. Prediction markets industrialise this. Instead of asking 787 people to guess once, they let thousands of people continuously revise their estimates, and they weight each revision by how much money the person is willing to put behind it. Someone who is confident and informed buys aggressively. Someone who is unsure trades small or stays out. The market price is a money-weighted average of beliefs, updated in real time.

This is why a contract trading at 73 cents implies a 73% probability. The mechanic is explained more fully in our guide to how prediction market odds work, but the headline is simple: prices and probabilities are the same number expressed in different units.

Skin in the game does most of the work

A pundit on television pays no price for being wrong. A pollster's reputation suffers in aggregate, slowly, over years. A trader who buys YES at 80 cents on a contract that resolves NO loses 80 cents per share, immediately and visibly.

That asymmetry is the engine. Markets force participants to convert vague opinions into specific prices, and they punish bad prices with real money. The economist Robin Hanson, who designed the prediction market mechanism used at firms like Google and Ford, has a line for this: markets do not reward sounding smart, they reward being right. A trader who buys a contract at 60 cents because they genuinely think the true probability is 75% is making money in expectation. A trader who buys at 60 cents because they want their team to win is donating to the people on the other side.

Over enough trades, the second type loses their bankroll and stops trading. The first type accumulates capital and influence on prices. The market gets sharper over time as a function of who is left.

Brier scores, and why forecasters use them

If you want to test whether a forecaster is good, you cannot just ask whether they were right. Anyone can predict a 99% certainty on a near-certain event and look brilliant. The standard tool is the Brier score, named after the meteorologist Glenn Brier, who proposed it in 1950 to evaluate weather forecasts.

The Brier score is the average squared error of a probabilistic forecast. If you say there is a 70% chance of rain and it rains, your error is 0.30 squared, or 0.09. If it does not rain, your error is 0.70 squared, or 0.49. Lower is better. Perfect forecasters score 0. A coin-flip forecaster on every question scores 0.25.

Academic work comparing prediction markets to conventional forecasts has generally found markets hold up well. A 2004 survey by Justin Wolfers and Eric Zitzewitz in the Journal of Economic Perspectives concluded that market-generated forecasts are typically fairly accurate and outperform most moderately sophisticated benchmarks, including polls. They were not always right. They were less wrong, more often, in a measurable way.

Calibration is the test that matters

A market that prices an event at 70% should see that event happen roughly 70% of the time. Across a large sample of 70% predictions, you want the realisation rate to land in a narrow band around 70%. This is calibration, and it is the cleanest test of whether a forecasting system is honest.

Polymarket published a calibration analysis covering thousands of resolved contracts. Events priced between 65% and 75% resolved YES at a rate close to 70%. Events priced between 25% and 35% resolved YES at a rate close to 30%. The line was not perfect. It was straight enough that the platform's prices could be used as probabilities without serious adjustment.

And unlike a pundit's track record, good calibration is evergreen: it does not depend on a single analyst staying sharp, because the mechanism keeps re-pricing as new money and information arrive. The system, not the personality, is what holds up over time.

Where markets beat polls, and where polls still win

The comparison is not symmetrical. Our longer comparison of prediction markets versus polls walks through the trade-offs, and the pattern below holds across much of the literature, though not all of it: the 2024 study above is a substantial counterexample. Markets tend to beat polls on binary or low-cardinality outcomes (who wins, will an event happen by a date), on questions where new information arrives continuously, and on long-horizon forecasts where polls go stale. Polls tend to win, or at least match, on questions about precise margins, demographic breakdowns, and counterfactual scenarios where no money is at stake to attract informed traders.

A poll asks a representative sample what they currently think. A market asks anyone with money and conviction what they think will happen. The first is a snapshot of current opinion. The second is a forecast of future reality, which is what most readers actually want.

When markets get it wrong

Accuracy is not infallibility. Markets fail in four identifiable ways.

The first is thin liquidity. A market with $5,000 of total volume on an obscure contract is not aggregating much information. A handful of traders can move the price arbitrarily, and the implied probability is closer to a single person's opinion than a crowd's. Our explainer on liquidity in prediction markets covers when to trust a price and when to discount it.

The second is favourite-longshot bias. Across many markets, low-probability outcomes (priced at 5% or less) tend to resolve at slightly below their implied rate, and high-probability outcomes (priced at 95% or more) tend to resolve at slightly above. The biases are small, on the order of one or two percentage points, but they are persistent. Sophisticated traders price them in.

The third is information asymmetry. Markets on private corporate decisions, jury deliberations, or anything where a small number of people know the answer and most traders are guessing, behave more like noise than signal. In 2024, a France-based trader placed tens of millions of dollars on Donald Trump in Polymarket's US presidential market, acting on polls he had privately commissioned: surveys that asked voters in swing states who they thought their neighbours would back, designed to correct for what he believed was an undercount of Trump support. That trader is usually cited as proof the market was efficient, aggregating information nobody else had. He is also the clearest illustration of the fourth failure mode, and the same example belongs in both places.

The fourth is concentration, and it is distinct from information asymmetry in a way worth being precise about. Asymmetry is someone knowing more than everybody else. Concentration is a few participants holding enough of the book to set the price, whatever they know. The French trader was both at once: he had private information, and he had roughly $45 million across four accounts, which made him one of the largest holders on that side of the market. Remove the private polling and the concentration remains, because a price can be moved by a handful of large positions whether or not those positions are well informed. That is why a headline probability tells you nothing on its own about how many people agree with it, and why the study above found prices for identical contracts diverging across exchanges: a market thick with capital but thin with participants can be confidently wrong in a way a poll of the same subject cannot.

The strongest evidence for this does not come from a critic. Kalshi's chief executive, Tarek Mansour, told the New York Times in August 2026 that full-time forecasters are not a marginal presence on his exchange: against an assumed figure of around 2% of volume, he put them at "70 to 80 percent of volume". That is the operator of a major venue describing his own order flow as dominated by a small group, which is a stronger source than any outside estimate and a less comfortable one for the wisdom-of-crowds account.

But read the whole of what he said, because the obvious inference from that number is wrong. Mansour also said Kalshi's best inflation forecaster is a private individual in Kansas, and that the top ten political forecasters on the platform are ordinary people rather than institutions. So the claim is concentration WITHOUT elite capture: a small number of participants move most of the volume, and those participants are individuals doing a great deal of research rather than desks at banks. That is a genuinely different picture from either of the stories usually told about prediction markets. It is not thousands of independent views averaging out, and it is not institutional money quietly setting the price. It is a handful of obsessives, and whether that makes a market more or less reliable than a poll depends entirely on how good the obsessives are.

Why this matters for anyone using the prices

The practical use of all this is straightforward. When you see a contract trading at 62 cents, you are looking at a probability estimate that has been stress-tested by people willing to lose money on it. That estimate is not infallible, and it is not automatically better than the alternatives. What it reliably is, is faster: it updates continuously, while a poll, panel or pundit does not.

iPredicta is the discovery layer for that signal. We aggregate the major prediction-market venues (Polymarket, Kalshi, Betfair, Smarkets and others), surface the prices that matter, explain what they mean in plain English, and flag what is accessible from the reader's region. The accuracy of the underlying markets is what makes the whole exercise worthwhile. Without it, the prices would just be vibes with a decimal point.

Frequently asked questions

Are prediction markets really more accurate than polls?

Often, on binary political and event-driven questions, but not reliably and not by construction. Decades of calibration and Brier-score work is broadly favourable: the Iowa Electronic Markets came closer to the outcome than national polls about three-quarters of the time across the elections studied, and Polymarket's calibration on resolved contracts tracks reasonably close to the diagonal. Set against that, the largest recent study of political markets found the four major venues ranging from 93% beating chance on PredictIt to 67% on Polymarket in the final weeks of the 2024 US campaign, with prices for identical contracts diverging across exchanges. Markets do best on binary questions like who wins; they tie or lose on precise vote margins and demographic breakdowns, where polls are built to be granular. The honest summary is that a market price is a faster and usually well-calibrated forecast of an outcome, and a poll is a better measurement of current opinion, and that neither is a substitute for reading the liquidity and the concentration behind the number.

What is a Brier score and why do forecasters care about it?

A Brier score is the average squared error of a probabilistic forecast, proposed by meteorologist Glenn Brier in 1950 to grade weather predictions. Lower is better, zero is perfect, and a coin-flip forecaster scores 0.25 on every question. The reason it matters is that it punishes both overconfidence and underconfidence honestly. A pundit who says "definitely yes" and is wrong takes a huge hit. A forecaster who says "60% likely" and is wrong takes a smaller, proportionate hit. Brier scores let you compare prediction markets directly against expert panels, pundits, and polling aggregates on the same numerical scale, which is why academic work on market accuracy leans on them so heavily.

What does calibration mean for prediction markets?

Calibration is the test of whether a market's stated probabilities match how often events actually happen. If a platform prices contracts at 70% and those contracts resolve YES roughly 70% of the time across a large sample, the market is well-calibrated. Polymarket's published calibration data on thousands of resolved contracts shows the realisation rates tracking close to the implied probabilities across the range, with small deviations at the extremes. Good calibration is what lets you treat a market price as a probability for practical decisions. Poor calibration means the prices are systematically biased and need adjustment before you act on them.

Why does money on the line make markets more accurate?

Skin in the game forces participants to convert opinions into specific prices and punishes bad prices with real losses. A pundit on television pays no cost for being wrong, so their incentive is to sound confident and entertaining. A trader who buys YES at 80 cents on a contract that resolves NO loses 80 cents per share immediately. Over time, traders who consistently misprice events lose their bankroll and exit the market, while traders who price events accurately accumulate capital and influence on prices. The system selects for accuracy in a way pundit ecosystems and polling firms do not, because the feedback loop is direct, financial, and fast.

When are prediction markets unreliable?

Markets fail in three predictable ways: thin liquidity, favourite-longshot bias, and information asymmetry. A contract with only a few thousand dollars of volume is not aggregating much information and can be moved by a single trader, so the implied probability is closer to one person's opinion than a crowd's. Extreme probabilities (under 5% or over 95%) show small but persistent biases where longshots are slightly overpriced and favourites slightly underpriced. And markets on private decisions, jury verdicts, or insider-driven outcomes can be efficient about information most retail traders cannot see, which makes the price misleading for anyone reading it as a public signal.