A Many-Designs Study of 516 Algorithms Challenges the Predictability of Wisdom of the Crowd Forecast Aggregation

König-Kersting, Christian; Litovsky, Yana; Böhm, Robert; Grossmann, Igor; Huber, Jürgen; Kirchler, Michael; WoCCAP Consortium

Working paper2026Expectations and forecasting

A Many-Designs Study of 516 Algorithms Challenges the Predictability of Wisdom of the Crowd Forecast Aggregation

Christian König-Kersting, Yana Litovsky, Robert Böhm, Igor Grossmann, Jürgen Huber, Michael Kirchler and WoCCAP Consortium (One of the research teams in the WoCCAP Consortium; comments on and editing of the manuscript)

University of Innsbruck Working Paper 2026-05

PDF RePEc Cite

A schematic, not data: one dot per algorithm. No algorithmic or researcher characteristic consistently explained accuracy, and researchers overrated how well their own and others' algorithms would do.

In short

129 research teams submitted 516 algorithms to pool the same forecasts. Hardly any approach beat the others in a predictable way.

What we find

129 teams independently built 516 aggregation algorithms, applied to forecasts by 1,182 people in economics, politics, climate and sports.
Most approaches did not predictably outperform the others.
The exception: algorithms that learned from past outcomes did better in politics and climate.

Cite

Christian König-Kersting, Yana Litovsky, Robert Böhm, Igor Grossmann, Jürgen Huber, Michael Kirchler and WoCCAP Consortium (2026). “A Many-Designs Study of 516 Algorithms Challenges the Predictability of Wisdom of the Crowd Forecast Aggregation.” University of Innsbruck Working Paper 2026-05.

@techreport{koenigkersting2026many,
  author = {König-Kersting, Christian and Litovsky, Yana and Böhm, Robert and Grossmann, Igor and Huber, Jürgen and Kirchler, Michael and {WoCCAP Consortium}},
  title = {{A Many-Designs Study of 516 Algorithms Challenges the Predictability of Wisdom of the Crowd Forecast Aggregation}},
  year = {2026},
  institution = {Faculty of Economics and Statistics, University of Innsbruck},
  type = {University of Innsbruck Working Paper},
  number = {2026-05}
}