Can the Wisdom of the Crowds predict the results of the 2018 World Cup?

Given the interest in the event around the world, we decided to conduct an experiment to see if we could predict various results of the tournament. We used a concept known as the Wisdom of the Crowds to see how accurately a group of people could predict certain outcomes of the World Cup in various statistical categories such as, goals scored, yellow cards given, red cards given, etc. But before we get into the results, let’s look at the concept of the Wisdom of the Crowds.

What is the Wisdom of the Crowds?

The basic idea of the Wisdom of the Crowds is that the average of the answers of a group of individuals is often more accurate than the answer of any one individual member of the group. Essentially, if you were to poll a group of people on their predictions for a certain event, the likelihood that the answer of a single member of the group is more accurate than the average answer of the whole group, is well, not very likely at all. The Wisdom of the Crowds is therefore used in many different fields, such as economics, politics and psychology because of its propensity to predict accurate results.

Perhaps the best way to explain how it works is to tell the story of Sir Francis Galton, a Victorian polymath, who was the first to note the Wisdom of the Crowds at a livestock fair he visited in 1906. Attendees of the fair were given the opportunity in one contest to guess the weight of an ox. The closest to the actual weight would win a prize.

There were over 750 participants to try their hand at guessing the ox’s weight. After all of the guesses were tallied, the interesting thing to note was that nobody got the exact weight of the ox spot on, but when Galton took all of the guesses from each participant and calculated the mean average, incredibly, the average turned out to be the exact weight of the ox: 1,198 pounds.

This is essentially how consensus forecasts work in economics, and it is the way FocusEconomics elaborates its Consensus Forecasts, leveraging the idea of the Wisdom of the Crowds. We collect, process and analyze projections from our global network of over 1000 trusted economic experts and ultimately condense that information to provide one answer, the FocusEconomics Consensus Forecast. 

So, what does this have to do with the World Cup? We wanted to test if the Wisdom of the Crowds to predict various results of the World Cup. We polled over 200 economic analysts from our network of global analysts to get their predictions, and then we averaged them all out after the tournament had ended to see if the Wisdom of the Crowds was indeed able to predict the results. Spoiler alert, not everything went to plan, however, some results were almost perfect. We’ll explain that shortly.

Results

Before we get into the results, we should address the big change to the 2018 FIFA World Cup that significantly skewed some statistical categories from their historical ranges and averages: video assistant referee (VAR).

VAR is a system that assists the referee of a game to make better decisions. A team of referees and assistants are located in a centralized location with access to all camera angles inside the stadium and advise the referee on situations that warrant review or that they believe the referee on the field may have gotten incorrect.

Although video replay technology has been employed to help officiating crews in many sports for years now, football (known in the U.S. as soccer) is one of the laggards in this respect. Some domestic club leagues around the world have adopted it in the last few years, but many still have yet to do so, including the most viewed league in the world, the Barclay’s Premier League. However, it was confirmed shortly before the tournament that VAR would be used for the first time at the 2018 FIFA World Cup.

It’s important to note that VAR was only used in what FIFA deemed “game-changing” incidents. These incidents were goals, penalty decisions, direct red cards and cases of mistaken identity. And this is exactly what brings us to the skewed statistics mentioned previously, the number of red cards, penalties given, and logically, penalties scored, were skewed drastically from the historical ranges of all of the previous World Cups.

What wasn’t really impacted by VAR significantly were the various categories associated with goals (apart from penalties scored) and yellow cards issued. Although goal scoring situations were one of the incidents that VAR was supposed to help evaluate, there were very few instances where a goal was either allowed or disallowed after a VAR review.

With that said, you can see below the results of our experiment with the Wisdom of the Crowds in predicting outcomes at the World Cup:

Conclusion

To conclude, the Wisdom of the Crowds works remarkably well most of the time, however, it’s not always perfect, but predicting the future is not an easy thing to do. Using a consensus to predict the outcome of a certain event is a pretty good way of coming to a more accurate conclusion. What the consensus does more often than not is reduce the risk inherent with forecasting events, but there is almost always going to be some margin of error and that has to be kept in mind when predicting the future.

Further Reading:

If you’d like to know more about how the Wisdom of the Crowds and Consensus Forecasts work to produce more accurate forecasts, check out the articles and academic research below:

• The Wisdom of the Crowds and the Consensus Forecast
• Consensus vs. Individual Forecasts: The Numbers Don’t Lie

 

“Consider what we have learned about the combination of forecasts over the past twenty years. (…) The results have been virtually unanimous: combining multiple forecasts leads to increased forecast accuracy. This has been the result whether the forecasts are judgmental or statistical, econometric or extrapolation. Furthermore, in many cases, one can make dramatic performance improvements by simply averaging the forecasts.”

Clemen Robert T. (1989) “Combining forecasts: A review and annotated bibliography” International Journal of Forecasting 5: 559-560
 

“A key reason for using forecast combinations […] is that individual forecasts may be differently affected by non-stationaries such as structural breaks caused by institutional change, technological developments or large macroeconomic shocks. […] Since it is typically difficult to detect structural breaks in ‘real terms’, it is plausible that on average, across periods with varying degrees of stability, combinations of forecasts from models with different degrees of adaptability may outperform forecasts from individual models.“

Aiolfi M. and Timmermann A. (2004) “Structural Breaks and the Performance of Forecast Combinations”: 2