Umpires have a 'mercy bias.' Could judges, too?

In his Senate confirmation hearing in 2005, the aspiring chief justice, John Roberts, offered a memorable analogy: "Judges are like umpires. Umpires don't make the rules; they apply them. The role of an umpire and a judge is critical. They make sure everybody plays by the rules."

The analogy was effective, because many people want judges to be umpires, dedicated to neutral application of the rules. In baseball, however, many fans don't agree that umpires are objective. Are they right?

We are obtaining a lot of data on that question, and a new paper finds some identifiable biases on the part of baseball's umpires. The biases are fascinating, not only because of what they tell us about baseball, but also because of what they tell us about human nature, and perhaps a general human propensity toward mercy.

To test for bias, Stanford University's Etan Green and David Daniels studied more than 1 million calls made by umpires from 2009 to 2011.

Not surprisingly, they found that some pitches are almost 100 percent likely to be called strikes, while others are almost certain to be called balls. But there is an area of uncertainty around the edge of the strike zone. That's where the bias occurs. Green and Daniels conclude that despite "their professional directive and expertise, umpires err in their decisionmaking; their mistakes are systematic, sizable and pervasive."

To test for bias, the authors examined how the umpires' judgments about the size of the strike zone change in four conditions: when the count has two strikes, when the count has three balls, when the preceding pitch was a ball and when the preceding pitch was a called strike.

When the batter has two strikes, the strike zone turns out to get a lot smaller. Apparently, umpires don't like to call a third strike. Similarly, the strike zone gets much smaller right after a called strike. These effects are remarkably large, producing "as much as a 20 percentage point drop, for the average umpire, in the probability of a strike."

These findings have clear implications for hitters, who should know that after a called strike, or with two strikes in the count, they're going to get the benefit of the doubt. And pitchers should know that in these circumstances, the umpire is likely to say "ball!" in the face of ambiguity.

Green and Daniels limit their analysis to their particular data, and they don't speculate about implications beyond baseball. But it's not unreasonable to think that they have uncovered a kind of "mercy bias." When hitters are vulnerable, umpires are strongly biased in their favor. They don't want to call a third strike or even two strikes in a row. Revealingly, there is little evidence of an "even-the-scales bias," because umpires don't expand the strike zone after a ball, and while an expansion of the strike zone does occur with a three-ball count, it is small.

Chief Justice Roberts thinks judges are analogous to umpires. If so, we have some obvious questions: Do judges show a general bias in favor of mercy? When they deal with criminal defendants who are at risk of imprisonment, do they contract the strike zone? And when parents and teachers are dealing with children, do they become merciful when children or students are at risk of striking out? Do employers behave similarly?

We don't have much data on these questions, but in time it's likely we will. There's good reason to speculate that we will find that human beings in general, and not just umpires in particular, tend to become more merciful when people have two strikes against them.

Is that a bias? Sure, we can see it that way. But we might also take it as a reflection of humanity's general agreement with the words of Shakespeare's Portia, literature's greatest advocate, who insisted that "earthly power doth then show likest God's/When mercy seasons justice." Baseball umpires follow Portia, and a lot of other people undoubtedly do so as well.