Sunday, April 27, 2008

Confirmation bias

From BPro's Kevin Goldstein:
Well sometimes prospects just explode, and small sample sizes sometimes are damned, as here is one player from each of the full-season leagues who is exploding, but also has the scouting reports or existing potential to confidently up their stock.
Wrong. Wrong. Wrong.

Wrong, wrong, wrong, wrong, wrong, wrong, wrong.

This is classic confirmation bias. "I thought these guys were going to be good, and, hey, they are good." Small samples are always small samples. They do not add anything to the discussion.* These guys may still be excellent prospects, but that has nothing to do with their performance in a limited set of games and everything to do with the fact that they were already great prospects. This mindset has absolutely got to be stricken from baseball coverage.

* OK, technically they add a small sliver of evidence. If a prospect was 90% likely to be awesome, the small sample tells us that he is now 90.01% likely to be awesome.** If you want to hitch your analysis wagon to that, that's your business.

** Numbers are totally made up for example purposes. You get the point, I hope.

No comments: