?

Log in

No account? Create an account
Long-shot Prediction Game: early probabilities - Many a mickle maks a muckle

> Recent Entries
> Archive
> Friends
> Profile

July 22nd, 2008


Previous Entry Share Next Entry
04:26 pm - Long-shot Prediction Game: early probabilities
Thank you to the twenty players who made submissions for Phase 1 of the long-shot prediction game. It probably needs a slightly better name at some point.

Of the twenty submissions, sixteen were perfect in every way. One had five nominations for "most likely", which is easy to deal with; we'll just lop the bottom one in the list off, Procrustean stylee. A second was missing a numerical nomination for how many were likely to occur. A third only had one nomination for "least likely to occur" and a fourth only had three nominations for "least likely to occur". Sadly there was some overlap between the latter two, otherwise I might have cobbled together a nineteenth submission of "four least likely"s. As it is, I fear it's probably best to ignore both incomplete submissions. I fear that incomplete entries must be scored as pessimistically as possible.

From the 19 entries that submitted nominations for "How many of the twelve events will occur?", the mean number was 4.052. As this suggests that the typical opinion was 4.052 of the 12 events would occur, this tends to suggest that the consensus opinion was that the average event on that list was 4.052/12 = 33.77% likely to occur. Amending submitted answers as discussed, and folding in the two sets of submissions received by e-mail, the conclusion (after rounding to the nearest 1%) from the 20 sets of "four most likely" and the 18 sets of "four least likely" is as follows:

Oil to trade at $200/barrel or higher.
70% say "in the four most likely", 22% say "in the four least likely"

USA to win most gold medals in Beijing.
60% say "in the four most likely", 6% say "in the four least likely"

HP 6 to gross at least $285 million at the US box office.
55% say "in the four most likely", 17% say "in the four least likely"

BBC Global 30 stock market index to drop below 4750.
45% say "in the four most likely", 6% say "in the four least likely"

Microsoft to takeover Yahoo.
35% say "in the four most likely", 28% say "in the four least likely"

USA and/or Israel to execute an overt air strike against Iran.
35% say "in the four most likely", 39% say "in the four least likely"

Gordon Brown to stop being Prime Minister of the UK, Labour Party leader or both.
25% say "in the four most likely", 33% say "in the four least likely"

No more than 2 major hurricanes in 2008 Atlantic Hurricane season.
25% say "in the four most likely", 56% say "in the four least likely"

Firefox usage to reach 33% or higher.
20% say "in the four most likely", 39% say "in the four least likely"

Prince William, Prince Harry, Princess Beatrice and/or Princess Eugenie to marry publicly.
15% say "in the four most likely", 50% say "in the four least likely"

New England Patriots to win Super Bowl XLIII.
10% say "in the four most likely", 39% say "in the four least likely"

John McCain to win at least 336 Electoral College seats.
5% say "in the four most likely", 78% say "in the four least likely"

Now here comes the arbitrary, faffy bit, which makes stage two of the process necessary. If the panel agrees that the average event is 33.77% likely to occur, those events most frequently named as among the four most likely to happen presumably have a higher chance of occurring than that. Conversely, those events most frequently named as among the four least likely to happen presumably have a lower chance of occurring than that. Trying to work out how much to finesse this 33.77% chance in each case is the tricky bit.

I am going to use an extremely naive metric based upon percentage saying "among the four most likely" minus percentage saying "among the four least likely". This does not distinguish between an event where nobody says "among the four most" or "among the four least" and an event where half say "among the four most" and half say "among the four least", but as we're more interested in a single point estimate of probability rather than any degree of confidence in the point estimate, this is as reasonable a starting-point as any. Unless, of course, any of you have thoughts about designing a better one to use next one.

The panel have collectively concluded that first estimates of the probabilities of the twelve long-shot events occurring after July 31st, 2008 and before February 1st, 2009 (unless otherwise stated) are:
USA to win most gold medals in Beijing.                            52%
Oil to trade at $200/barrel or higher.                             50%
BBC Global 30 stock market index to drop below 4750.               47%
HP 6 to gross at least $285 million at the US box office.          46%
Microsoft to takeover Yahoo.                                       36%
USA and/or Israel to execute an overt air strike against Iran.     33%
Gordon Brown to stop being Prime Minister, Labour leader or both.  31%
Firefox usage to reach 33% or higher.                              28%
New England Patriots to win Super Bowl XLIII.                      25%
No more than 2 major hurricanes in 2008 Atlantic Hurricane season. 24%
Named British Prince or Princess to marry publicly.                23%
John McCain to win at least 336 Electoral College seats.           10%

You may note that these probabilities sum to 405%, roughly consistent with the mean opinion that 4.052 of them will happen. The second part of the process - and, I promise you, there are only three parts - is to use whatever extra information you may have received between now and then to reflect whether any of these probabilities are too high or too low. If about as many people say that a probability is too low as say it is too high, then it's probably about right. If there's much stronger support for one sentiment than there is for the other, then (assuming rational behaviour, as we are doing throughout, motivated by the abstract prospect of winning a game) the stronger-supported sentiment is probably more likely to be correct.

Particular emphasis will be given to people who have previously said that an event is among the four most likely to occur and then claim that the probability estimate is too high, or people who have previously said that an event is among the four least likely to occur and then claim that the probability estimate is too low. This sort of U-turn is, I would argue, more significant than the word of someone suggesting reinforcing their previous contention that a long-shot was either relatively likely or relatively unlikely to occur. (If you didn't participate in the first stage of the game, you're still welcome to submit your opinions.)

Given that I go back to work on Friday morning, and given the times it took participants to respond to the poll if they did so at all, then I'll seek to wrap this up on Thursday, probably in the early evening, British time. All events are shorthand for the ones described and clarified previously.
Poll #1227749 Long-shot Prediction Game poll, part two
This poll is closed.

Do any of the probability estimates look too high?

The probability of "USA to win most gold medals in Beijing" should be under 52%.
1(10.0%)
The probability of "Oil to trade at $200/barrel or higher" should be under 50%.
1(10.0%)
The probability of "BBC Global 30 stock market index to drop below 4750" should be under 47%.
0(0.0%)
The probability of "HP 6 to gross at least $285 million at the US box office" should be under 46%.
1(10.0%)
The probability of "Microsoft to takeover Yahoo" should be under 36%.
0(0.0%)
The probability of "USA and/or Israel to execute an overt air strike against Iran" should be under 33%.
0(0.0%)
The probability of "Gordon Brown to stop being Prime Minister, Labour leader or both" should be under 31%.
0(0.0%)
The probability of "Firefox usage to reach 33% or higher" should be under 28%.
0(0.0%)
The probability of "New England Patriots to win Super Bowl XLIII" should be under 25%.
1(10.0%)
The probability of "No more than 2 major hurricanes in 2008 Atlantic Hurricane season" should be under 24%.
0(0.0%)
The probability of "Named British Prince or Princess to marry publicly" should be under 23%.
0(0.0%)
The probability of "John McCain to win at least 336 Electoral College seats" should be under 10%.
0(0.0%)

Do any of the probability estimates look too low?

The probability of "USA to win most gold medals in Beijing" should be over 52%.
1(12.5%)
The probability of "Oil to trade at $200/barrel or higher" should be over 50%.
0(0.0%)
The probability of "BBC Global 30 stock market index to drop below 4750" should be over 47%.
0(0.0%)
The probability of "HP 6 to gross at least $285 million at the US box office" should be over 46%.
0(0.0%)
The probability of "Microsoft to takeover Yahoo" should be over 36%.
0(0.0%)
The probability of "USA and/or Israel to execute an overt air strike against Iran" should be over 33%.
0(0.0%)
The probability of "Gordon Brown to stop being Prime Minister, Labour leader or both" should be over 31%.
1(12.5%)
The probability of "Firefox usage to reach 33% or higher" should be over 28%.
0(0.0%)
The probability of "New England Patriots to win Super Bowl XLIII" should be over 25%.
0(0.0%)
The probability of "No more than 2 major hurricanes in 2008 Atlantic Hurricane season" should be over 24%.
0(0.0%)
The probability of "Named British Prince or Princess to marry publicly" should be over 23%.
0(0.0%)
The probability of "John McCain to win at least 336 Electoral College seats" should be over 10%.
3(37.5%)

(Leave a comment)


> Go to Top
LiveJournal.com