Bayesian Relief

by Hubertus Hofkirchner -- Vienna, 20 Jul 2016

Concurrent with technological progress and increasing globalisation, the world is moving faster and faster. Decision makers in companies and organisations must keep up with the accelerating pace or suffer the consequences. However, traditional market research methods simply take time until recruiting, screening, fielding and interpretation can be completed. It is not uncommon that their static results are outdated even before they become available.

This development coincides with consumers becoming increasingly individualistic. Their rapidly evolving preferences render segmentation and representativeness much harder than in the last decade. They also are fickle panelists, the 2015 GRIT report deplores ever lower response rates, carelessness and shrinking attention spans of respondents which are exacerbated by privacy concerns and reluctance to reveal their true thoughts and intentions.

New Technology

Caught between a rock and a hard place, researchers are turning to new technologies. One method shows particular promise in light of the speed challenge: second generation prediction markets. Right from their start in the late nineties, first generation systems produced uncannily accurate election forecasts, but proved to be too unwieldy for other types of research. Only the largest of clients could afford the significant system investment and no one really knew how to apply these big platforms properly to use their full potential.

Modern systems employ real-time trading engines which instantly update the results after each response. They react within seconds to changes in the market, just like the global stock exchanges. Delivered as flexible cloud services, they now come with a versatile arsenal of question types for a host of different probing purposes. Highly specialised providers have developed best practices for how to employ the technology for all kinds of projects. Meanwhile, their range of tried and tested applications is comparable to the menu of full-service traditional research providers.

The Differences

So, which properties of the new second generation systems are of particular interest to researchers in light of the growing challenges of the marketplace?

1. Speed & Agility -- The continuous Bayesian updating mechanism of second generation prediction market aggregates forecasts immediately which helps researchers to keep ever tighter deadlines. Once a market is set up, it can be kept alive indefinitely with only a moderate effort to retain its community of traders. Should conditions change or a client’s competitor make a strategic move, the prediction market can be reopened in no time. Trading will rapidly rebalance all forecasts to reflect the new situation. A long-term strategic research project thereby morphs into a rapid tactical barometer.

2. Integrating Quant & Qual -- Two parallel conversations happen on a modern prediction market. Orders and trades communicate quantified opinions, while traders’ market talk reveals their underlying reasons. The obvious advantage is that numeric KPI forecasts can be understood in conjunction with the coveted whys. Conversations, even when interspersed with irrelevant chatter, can be read in relation to market prices changing (or not) without extra effort by the researcher. This unique quantification of qualitative insights separates the proverbial signal from the noise.

3. Authentic Interpretation -- The virtual shares on a prediction market are the very KPI’s which clients hope to achieve. Instead of constructs of uncertain validity - like purchase intent, credibility or differentiation - a prediction market produces tangible commercial KPI forecasts which the client understands directly, such as future market share, sales volume or audience numbers. The traders themselves translate their thinking into numeric orders. This reduces the scope for incorrect interpretations and forestalls the dreaded confirmation bias.

4. Representativeness -- Prediction markets thrive on well-diversified, multi-faceted knowledge and informed opinions about the future. The number of minds which hold this information matters not as long as many diverse viewpoints are present. Experiments have shown that communities of sixteen traders for any single question are fully sufficient for good results. As a consequence, the frequently lamented decrease of representativeness in terms of conventional demographics ceases to be an issue. Consider a stock exchange where a strictly vegetarian trader may perfectly well trade with success in pork belly futures.

5. Democratisation -- As with all things internet, online markets facilitate a fundamental change, from one-way communication between researcher and respondent to a two-way dialogue with and between a new breed of empowered participants. The role of the researcher changes: he or she is the super-analyst who takes the many subjective predictions by analyst-respondents to the next level, he interprets the overall analysis and consults clients on the best ways to act on this information.

6. Gamification -- Instead of merely ticking checkboxes and then being left in the dark, participants get immediate feedback and experience a game-like flow. The real-time market gives them an instant sense of winning or losing as prices move for or against their beliefs, team spirit develops as contrarian opinions converge and a group consensus forms. Finally respondents can develop mastery from the ultimate feedback on how well they did, compared to real-world results.

7. Engagement -- Once responses and results from an external market are available, there is the option to duplicate a prediction market and its data for continued in-house trading. When staff participate in such an internal market, several benefits accrue. Results don’t just end up in a drawer, soon forgotten, instead participants consider them in a profound way before doing their own trades. The emerging forecasts align internal assumptions for better coordination. Everybody’s opinion is aggregated into the forecasts which fosters stronger commitment to achieve them. Clients also learn how the internal consensus differs from that of the external panelists.

8. Globalisation -- Electronic markets do not need a physical presence, research projects can be executed 100% digitally and remotely. They can be run from anywhere and at any time 24/7. Prediction questions always relate to objective hard facts or real events in the future, therefore cultural differences and nuances do not matter as much as with semantic constructs. In the market talk bilingual moderators can easily manage the local differences. Researchers can field multi-country studies centrally, with a consistently high level of execution quality and reporting.

What Comes Next

Currently small and mid-size traditional research firms are at a disadvantage to big companies which set the standard for fastest project execution. It stands to reason that the former will be the early adopters of second generation prediction markets to compete more effectively with the big firms.

There is a second possible future though, where veteran researchers resist innovation and new approaches. In this scenario, clients will commence to use prediction market tools directly in self-service mode, bypassing the traditional firms, small or big.

Whichever way this plays out, the old adage will apply: it is not the strongest who survive, it is those most responsive to change.

More by the Author:

• Corporate Innovators' Risk Reward Dilemma
• Swiss Democracy Needs to Catch Up to Internet Age
• Tapping Crowd Intelligence For Advert Efficiency Testing
• Liar, Liar: Who still believes in surveys?

Back to the Management by Predictions Blog
Follow this blog for more stories, new case studies and how-to's.