It must be true. A little bird told me.
A Strong Track Record?
At the time, it seemed reasonable to trust Cosmo – after all, he had correctly predicted the winners for all five games of the Western Conference final series between Vancouver and San Jose. The parrot clearly had the gift of foresight. After all, the odds of correctly picking all five winners by chance must be astronomical, right? Let’s see:
½ x ½ x ½ x ½ x ½ = 3%
Oh. Hmm. That doesn’t seem so astronomical, merely unlikely. And I guess, if I’m being honest, there might be some other factors (confounders) to consider, like:
- The fact that this experiment was conducted in Vancouver, introducing the possibility that the bird’s handler might have influenced the results in favour of the home team
- That there might be 31 other animals out there that didn’t get the prediction right.
The first of these is an example of experimenter bias — the phenomenon whereby the experimenter (in this case, the bird’s handler, who gave Cosmo the choice of team cards) unconsciously influences the outcome. But how can that happen if the handler doesn’t himself know the outcome?
Well it might just be coupled with hindsight bias, which Dan Gardner so eloquently describes as “Heads, I win. Tails, you forget we had a bet.” When the broadcast aired, we already knew that Vancouver had won the western conference – if Cosmo had been wrong, there would have been no story at all.
Hindsight bias also drives the second potential confounder. No one knows how many other animals predicted the outcome of the Western Conference, but if any picked the Sharks, they surely didn’t make it on the news.
Start with a not-all-that-unlikely probability that the bird just got lucky, add healthy servings of statistical, methodological, and cognitive biases, bake for 5 games and voila, a new expert.
The Stanley Cup Prediction
But the spotlight is cold, and once it’s on you, it can be hard to keep your mojo. Here are Cosmo’s picks for the Finals:
Now if you’re tempted to say, “Well, he only got one game wrong in the five he predicted. That’s pretty good,” go back and start this article from the beginning. Aren’t the odds of getting four out of five right better than his odds on the Western conference? And the fact that he picked the home team 4 out of 5 times – could that suggest experimenter bias?
Also: he’s an effing parrot. There is no way in the known universe that he can predict the future. Small issue.
Luckily for Cosmo, he’ll be forgotten soon enough. I’d be surprised if either CBC or CTV did a followup article – in fact, I wouldn’t be surprised if this is the only post-hoc assessment of Cosmo’s predictive capabilities that’s ever done. That’s one of the reasons hindsight bias is so powerful – we forget the misses as easily as we notice the hits.
Now don’t get me wrong: I’m not humourless. I get the entertainment value this type of story, and I don’t think most people put much stock in it. True, according to the CBC poll, 33% of people did not pick “No, are you kidding?” and thus presumably believe that the parrot may well know the future — but that’s not a statistically valid response, so we’ll just consider it part of the entertainment.
Rather, I’m writing this mostly because it’s a good illustration of how these biases work, and of course they’re not just limited to parrots. The CTV story also profiled the predictions of EA Sports who, we’re told, crunched the numbers to come up with a more scientific prediction. Here are their results:
That’s right, not a single correct score. Not a single correct OT prediction. And the Canucks win the series. I’m sure EA won’t be quite as eager to be featured in the press now that the results are known, and luckily for them, they probably won’t be.
For those who want to dig deeper, there’s a rich area of literature on expert predictions, and Gardner’s Future Babble is a great place to start. It’s a fascinating window into how our minds repeatedly fail to warn us that the experts are usually wrong – and often worse than trusting your dart board. The prediction game, it seems, really is for the birds.