Correct reasoning

So much of what I write here is really about correct reasoning with baseball as the example. This is because I love baseball and also want to reason about it (and everything else) correctly. Thus, this post, while not explicitly baseball related, naturally piqued my interest. Indeed, I saw it because it was linked by J.C. Bradbury over at Sabernomics, a baseball economics blog. The whole thing is well worth reading, but I will excerpt just a small amount.

If we don’t sometimes defer to the expert consensus, we’ll systematically tend to go wrong in the face of one-way-hash arguments, at least our own necessarily limited domains of knowledge.  Indeed, in such cases, trying to evaluate the arguments on their merits will tend to lead to an erroneous conclusion more often than simply trying to gauge the credibility of the various disputants. The problem, of course, is gauging your own competence level well enough to know when to assess arguments and when to assess arguers. Thanks to the perverse phenomenon psychologists have dubbed the Dunning-Kruger effect,  those who are least competent tend to have the most wildly inflated estimates of their own knowledge and competence. They don’t know enough to know that they don’t know, as it were.

Yes! Correct reasoning is hard. As much as I love it, I do it wrong all the time. Indeed, this blog serves to enshrine all the times I have reasoned incorrectly. I keep meaning to do a post summarizing all the analyses I've f***ed up, but I keep not doing it (insert Freudian crap about ego and id and all that jazz).

In any case, since we really don't have the time to reason correctly about every last thing, we absolutely must adopt the position that other people likely know more about any given issue and that we should generally defer to their judgement. Yes, this creates the problem of identifying who the experts are, but this is probably a more tractable problem.

The more I learn about computer programming or guitar or economics or baseball, the more I understand how little I truly know about any of them. This is why it is so important not to commit to a particular position just because you feel like you need to choose a side. You should never commit to a position until you have to, and even then you should commit to it only for as long as is necessary, at which time you should "uncommit" and continue to examine all evidence for every position. This is not easy to do.*

Finally, from the same post:

Addendum II: In the comments Pithlord lives up to his moniker:

Most fallacies aren’t really fallacies when you reinterpret them as Bayesian reasons to give an idea more credence rather than iron-clad syllogisms. Without the “argument from authority” and the “ad hominem fallacy”, you would either never get lunch or you’d give all your money to Nigerian spammers.

Bingo. Probabilistic reasoning is insanely hard. In propositional logic, it is a grave fallacy to say "A implies B; B is true; therefore A is also true." However, probabilistically, when you say that "if A is true, B is more likely to be true; B is likely to be true; therefore A is more likely to be true," you are 100% right. That's messed up.

* Please note that I am not saying that no position is ever right or wrong. Rather, I am saying that knowing which position is right and which is wrong can only occur with increasing degrees of certainty. Nor am I arguing that one should never advocate for a position or argue a particular viewpoint. Indeed, if we did not do this, we would never be able to communicate positions or viewpoints to people that need to hear them. Rather what I am saying is that the appropriate attitude when advancing a position is to remember that you could be wrong and to try to learn how you could be wrong from the opposing position.

Bill Simmons, Product of the Enlightenment?

I'm gonna start this post with two asterisks: (*), and (**). Now, onward!

Bill Simmons has a great article up on ESPN right now. Strangely, it's great from an analysis point of view and not a humor point of view (some of you might have found either one strange). Bill, known for his almost fanatical devotion to the idea of chemistry in football and basketball, slams the idea that chemistry is important in baseball.

But baseball … baseball is an individual sport masquerading as a team sport. You are a worker bee. You have a job, and it's up to you to execute it. Yeah, it's always better to get along, but couldn't you say that about any work situation? Ultimately, it's just you. You're the one pitching, you're the one hitting, you're the one fielding. If everyone is pulling for one another, fantastic. You can even win a division that way -- good karma invariably leads to goofy bounces and luck. On the other hand, the deliriously happy post-Manny Red Sox mustered just three hits in their biggest game of the season (Game 7, 2008 ALCS). At some point in baseball, talent trumps all.

I am in near complete agreement with this (not so much the karma part). Football and basketball are both sports in which teammates must actually cooperate to win. The idea that the relationship of teammates to each other is important is not that far fatched. In baseball, guys have to cooperate to turn a double play or make a relay throw or something. There just aren't that many ways in which good or bad chemistry can have an effect.

I never thought I'd be linking to Bill Simmons here. Weird.

* Did I really not post at all in March? I'm shocked (but not, you know, shocked, shocked).

** My apologies to those who don't get the post title. There are maybe four people in the world who really do.

**EDIT** Fixed a weird typo.

Is VORP dead?

I've said a few times that one of the things I would love from folks who hate VORP is an honest critique of it from a scientific and analytic perspective. Naturally, those who hate VORP have not done this, because to do so would legitimize the real thing that they hate: the use of science to describe something that they believe transcends such vulgar quantifications. This is nonsense, and we don't need to get into it again.

Of course, one of the hallmarks of science is that the old is always being discarded in favor of the new. Indeed, VORP has come under some criticism recently, but not the banal criticism you've come to expect from the neanderthal luddites that have a stranglehold on print, radio, and television. Rather, it's been the subject of intelligent criticism from people who understand not just how VORP works, but how science works.

The bottom line is that VORP is flawed. You can read a good rundown of its flaws here. Essentially, VORP incorrectly calculates positional adjustments and underestimates the impact of walks (ironically, it has been noted) and doubles. These are fixable problems, of course, and it may be that VORP will be adjusted in the future to account for them. Baseball Prospectus has already started to overhaul WARP to adjust for some of its deficiencies.

In any case, using VORP is still miles better than falling back on the classical trifecta of batting average, home runs, and RsBI (for you, Cous). However, just because it was quick to arrive on the analytical scene or because it's promoted by the biggest name in sabermetrics doesn't mean that deserves to stick around.

I haven't been hitting the stats too hard recently on this site. If and when I get back to it, don't be surprised to see VORP supplanted by a statistic with a better run estimator and positional adjustments. There's no reason to become attached to a particular statistic when we have the tools to progress past it.

Tim Raines: Master Thief

Poaching from Tom Tango again:

Tim Raines, in high-leverage situations, has reached 1B or 2B 755 times (1B+2B+BB+HB+ROE), and has 260 SB, 49 CS.  That’s 0.34 successful SB per opportunity.

Tim Raines, in low-leverage situations, has reached 1B or 2B 1312 times, and has 134 SB and 15 CS.  That’s .10 SB per opp.

Do you see that?  He stole a base 10% of the time in low-leverage situations, but 34% of the time in high-leverage situations.

So, to all those BBWAA writers who said that he didn’t steal enough… yeah, he didn’t pad his SB totals when the game situation didn’t matter. 

Would you like player who successfully steals a base 34% of the time when it really matters? while only making an out 6% of the time? Yes?

Why is this man not in the Hall of Fame?

Properly interpreting projections

Every year, right about now, you see two related phenomenon:

1. Fans completely misinterpreting the predictions of objective projection systems.
2. "Experts" releasing hilariously skewed subjective predicitons.

What am I talking about? Specifically this: people, whether interpreting objective projections or making their own subjective projections, fail to discern the difference between saying that no individual team is likely to win 100 games and that the league is likely not to see any team win 100 games; the difference between not projecting a single pitcher to win 20 games and projecting the league not to have any 20 game winners; the difference between projecting that no single hitter is likely to hit 40 homeruns and drive in 120 runners and projecting that the league is likely not to see any hitter hit 40 homeruns and drive in 120 runners.

Do you think all these things are the same? If you do, you are failing to understand a basic concept in probablility. There's no shame in this. Probability is freaking confusing. Unless you're really, really well-versed in it, you're going to screw it up all the time. I know I do. Nonetheless, let's dig a little deeper into this problem.

Let's say I have a game of chance. You roll a 20-sided die. If you roll a natural 20, I will give you twenty dollars. If not, you will owe me one dollar. What are your expected winnings? Well, we would project you to owe me one dollar 95% of the time and win twenty dollars 5% of the time. That means your expected winning from any single dice roll is exactly five cents (0.95 x -1.0 + 0.05 x 20.0). There's no ifs, ands, or buts about it. We would project you to win five cents.

But what if 1,000 people play my game? Since the dice rolls are independent, we would project each of them to win five cents, just like in the individual game. But does this mean that we expect no one to win twenty dollars? Of course not! In fact, we would expect 50 people to win twenty dollars, we just can't predict which 50 people will win. Thus, each person expects to win five cents, even though roughly 50 people will win twenty dollars. In fact, the odds of not one person winning twenty dollars are less than one in 10,000,000,000,000,000,000,000.

So what good is our expectation of a five cent win? Simple: that's the expectation that will give us the least error over an infinite number of games. We can't improve on it because the only thing we haven't accounted for in our model of expectation is random chance. Note that this is different from the most likely outcome for one game. The most likely outcome from one game is that you lose a dollar. Nonetheless, we can't just ignore the (relatively) massive twenty dollar payout, even if it is (relatively) rare. That's how expected value works. It aggregates all possibilities into one number that has properly weighted all outcomes.*

The situation appiles exactly the same way to baseball predictions. Some pitchers are going to win 20 games (probably), we just don't know which ones. Some teams are going to win 100 games, we just don't know which ones. Some hitters are going to hit 40 home runs and drive in 120 runners, we just don't know which ones. 

That's why when fans react negatively to a projection system that predicts no pitcher to win 20 games, they are making a big mistake. And it's also why when you see Joe Expert predicting some team to win 100 games or some pitcher to win 20 games, he's making a big mistake. Yes, these events are probably going to happen, but it's a fool who thinks he can pick exactly which player or team is going to do it.

There are too many variables that go into a baseball season to determine which teams and players will hit the highs and lows. That's why any sane projection gives numbers that fit in a much more narrow range than you will end up seeing in the real season. There's nothing wrong with this. There's nothing else you can do. If all that remains is random noise, then it will be impossible to improve on your projection's accuracy.** If you could eliminate it, it wouldn't be random would it?

If you can, it often helps to look at player and team projection in terms of percentiles instead of mean expectation. This helps you get a sense of the uncertainty in the projetion. You can more easily see highs and lows and see a variety of outcomes. However, if all you're looking at is the mean expectation, try to keep in mind my simple dice game. Even though you won't have the shape of the curve involved, you'll still understand that this is the expectation that will give you the least error when the real results finally come in.

* Note that expected value is not necessarily the right variable to use for decision making. Value and utility are not necessarily the same thing. Furthermore, it is not true that the utility of the expected value of a particular choice is the same as the expected utility. For both baseball and my dice game, the two are likely to be close enough to be interchangable. For more information, read about the St. Petersburg paradox (which is awesome, by the way).

** That's not to say that any projection system out there is truly left with nothing but random noise. There may be opportunities for genuine improvement, but these opportunities can never fully overcome randomness. Furthermore, these improvements must be derived through rigourous statistical and scientific processes, not someone's gut feeling or some arbitrary pattern they've pulled out of thin air. These are not improvements.

Andre Dawson and why one must consider the whole picture

Tom Tango has a great post up here on Andre Dawson and his qualifications for the Hall of Fame.

The point is that the low OBP for Dawson can’t be looked at in isolation, as something he has to overcome.  It’s one piece of the puzzle, that’s all.

This is spot on, and it's something that I've been dwelling on over the past few months (and not just with respect to baseball).

We have a tendency as humans, excelling as we do at pattern recognition, to prefer discrete classifications to continuous valuations. This is a mistake most of the time. There isn't some hard and fast line at which an on-base percentage becomes acceptable for a Hall of Famer. Any low on-base percentage can be made up for by sufficiently greater production in another area.

It's so easy to get caught engaging in this kind of analysis. We think "Player X can't possible be productive! He only gets on base 32% of the time and he's a singles hitter!" We may be right: Player X may not be productive. However, if Player X is also the best defensive shortstop in the league, there's a great chance that he is actually a highly valuable player.

At the same time, simply being the best defensive shortstop in the league doesn't automatically make Player X valuable. If his offense is poor enough, it doesn't matter that he's better at defense than his peers. In order to perform a proper valuation of Player X we need to quantify every single factor that we can and then see where our evaluation fits within the continuum of baseball players.

Now this isn't an excuse to fudge the numbers. Far from it. Just because we need to whole picture to make correct valuations doesn't mean we can suddenly start assigning arbitrary value to things like "intangibles" or "hustle" or "grit" or "heart." We still need to place analysis within the proper scientific framework.

We need to realize that things rarely fall into discrete buckets. When we bucketize continuous data, we leave ourselves open to error. That is the lesson.

The most wonderful time of the year

That's right! Spring is here! What's that you say? Haven't seen a robin hopping around in your yard yet? Never fear, fellow spring lover! An even more sure sign of the end of winter is upon us. Yes, baseball players have reported to camp, and virtually all of them are in the best shape of their lives!

Julio Lugo

Julio Lugo, who came in about 5 pounds heavier (seemingly all muscle) than he did last season, said he would have played in the World Series last fall had the Sox gotten that far. He was healthy by that point, and now is ready to begin the 2009 season, completely healed from the quadriceps injury that cut short his 2008 campaign. He's also, he said, in the best shape of his life.

Ryan Ludwick

Ludwick soon expressed remorse for what he said, although he added that he was in the best shape of his life after an active winter of weight training.

Juan Pierre

"That stuff is out of my hands," said Pierre. "I learned last year I can only control what I can control. I came in the best shape of my career and I'm ready to go. After last year, they can't throw too much stuff at me I won't be ready for. Everybody knows I want to play every day, but it's not in my hands."

Dustin Pedroia

Pedroia said he gave up ice cream over the winter as part of his training program and is in the best shape of his career. He said he's ready to help the team get back to the playoffs after losing to the Tampa Bay Rays in Game 7 of the American League Championship Series last October.

Ronnie Belliard

Bowden said Belliard is in the best shape of his career since he joined Washington two years ago. Belliard played in 96 games last year and hit .287 with 11 home runs and 46 RBIs. [John: WTF? Best shape of career since two years ago? What does that even mean?]

Brad Penny

Penny, who is younger than A.J. Burnett and in 2007 was third in the NL Cy Young Award balloting, is in the best shape of his career. He says his shoulder was so weak last year, "I never should have tried to pitch." But after a winter of dogged conditioning and buying into the Red Sox program, Penny says he is throwing "as well as I ever have" and pitching coach John Farrell now believes Penny will open the season in the rotation.

Freddie Sanchez

"I feel like I'm in the best shape of my career," Sanchez said. "This is the strongest I've ever been. There have been no setbacks in my throwing program. My shoulder is feeling good. It feels strong and I want to see where it all goes this season."

Ryan Howard

Howard's friends have been buzzing all winter about how committed he was to improving his defense and conditioning. And while it's a little early to assess his leather craftwork, there's no doubt he's in the best shape of his career, after dropping 20 pounds in the offseason.

Chris Lubanski

"I know I can run," he said. "I know I haven't run as much as I should have in the past. I've focused this offseason to get into the best shape of my life. That is one thing this year I'm going to show is my speed cause I know it is still there. I know I still have it."

Alex Gordon

He’s been a disappointment his first two years in the majors, but reports are he’s in the best shape of his life, and working with former Royals 3B Kevin Seitzer can’t hurt. 

Carlos Pena

"When I tell you I was heartbroken, I was heartbroken, man. I was like, 'You've got to be kidding me.' I waited for [the World Baseball Classic] two years, I couldn't be a part of this last time. Now I'm in it, and I can't be a part of it. But at the same time, I'm making sure I take advantage of this time to heal. And I think it will all work out for the best. So I haven't spent too much time dwelling on the fact I won't be a part of it. Instead, I'm like, 'All right, I've got a lot of time to get into the best shape of my life.'"

What's amazing about this list is that it covers the entire spectrum of MLB players. For Pete's sake, last year's AL MVP says that this year he's in the best shape of his life. I have bad news, Dustin: you aren't likely to improve on last year.

Again, I have no doubt that the players believe this and that for some of them it's actually true. What kills me is that writers bother to use it and unironically at that. We get it. You worked hard during the offseason. Now let's talk about something interesting!

Also, here is a special bonus "best shape" article:

[Padres relief pitcher Heath] Bell said Monday that his workouts with the Nintendo Wii Fit were a big reason why he dropped about 25 pounds during the offseason and why he considers himself in "tip-top" shape as he prepares for his role as closer now that icon Trevor Hoffman has moved to Milwaukee.

As for the Nintendo Wii, Bell often played that with his wife and children at their home in Florida. It wasn't until the Bells purchased the Wii Fit game and board that he started doing the exercises that helped him get in what he considers the best shape he's been in since joining the Padres in 2007. 

Woohoo! Happy baseball to all!

PS: Thanks, Google News!