Umpires vs. PitchFx Game Day Data

The post The Automated K Zone & Technology. Is it time? generated a discussion of how good/bad are the umpires at calling balls and strikes. So lets take a look at that question using the Game Day data which is presumably the same data used by the  BrooksBaseball.net PITCHf/x Tool.

Pitch Result Count
Automatic Ball 5
Ball 232,520
Ball In Dirt 15,501
Called Strike 118,755
Foul 119,351
Foul (Runner Going) 2,307
Foul Bunt 2,413
Foul Pitchout 1
Foul Tip 5,970
Hit By Pitch 1,602
In play, no out 30,183
In play, out(s) 84,847
In play, run(s) 15,500
Intent Ball 3,348
Missed Bunt 516
Pitchout 238
Swinging Pitchout 2
Swinging Strike 63,684
Swinging Strike (Blocked) 5,834

The table at right contains the counts for how each pitch in 2015 was categorized in the Game Day data.

In order to use this data we will have to make some assumptions and will also have to exclude some rows. Lets start with considering some measures of agreement:

  • How often does the umpire agree with the PitchFx classification for a pitch that is called a strike. The Called Strike row (118,755 pitches) represents pitches that were called strikes by the umpire.
  • How often does the umpire agree with the PitchFx classification for a pitch that is called a ball. The Ball row (235,520) are pitches that the Home Plate umpire called balls. Note that the Ball In Dirt pitches were excluded as those are likely not of interest (easy to call??).
  • For purposes of comparision, the following pitch classifications were also excluded as no umpire judgment on a pitch being a ball or a strike is required: Automatic Ball, Foul Pitchout, Hit By Pitch, Intentional Ball, Pitchout, Swinging Pitchout.

ZoneMap The pitch data includes a field called Zone. Zone values of 1 thru 9 represent strikes. The Zone values are from the perspective of the umpire. So Zones 1, 2 and 3 are the top of the strike zone and Zones 7, 8 and 9 are the bottom. Zone 5 is probably where a pitcher never wants to throw a pitch to anyone. Any other Zone values are, according to the Game Data Pitch data, balls.

Using this info we can create a few metrics:

  • What percent of Called Strikes by the umpire are not in Zones 1 thru 9.
  • What percent of Balls called by the umpire are in Zones 1 thru 9.
  • Given the exclusions listed above, all the other pitches are ones the batter swung at. So lets combine all the rows where the description is Missed Bunt or begins with Foul, In Play, or Swinging (except for the Pitchout one as that was previously excluded) as those that were swung at by the batter and calculate what percent of those pitchers were actually balls. The idea here is that for umpires that have big (or unpredictable) strike zones, batters are forced to swing at anything close. So this might be an indicator of umpires influence on batters. Clearly it also includes players who swing at bad balls (some on purpose), but lets assume that no umpires have disproportionately more games with lots of bad ball swingers.

The table below show for the 87 umpires who had at least 2500 pitches during 2015 (this cutoff removed just 5 umpires) how they rated on each scale.  For example, lets consider the first umpire listed Adam Hamari.

  • Only 24 umpires were better at not mistakenly calling a strike on a pitch that pitch fx thought was a ball.
  • Only 6 umpires were better at not calling a strike on a pitch that was a ball.
  • He was middle of the road in terms of batters being induced to swing at balls.

And to clarify and highlight Batter Swung At A Ball, consider Will Little (last row in the table).

  • He is among the best in calling balls as strikes. He ranks highly in terms of not calling a pitch that is a ball as a strike, and not calling a pitch that is a ball as a strike.
  • Yet he has a very high percentage of pitches that are balls that batters swing at.
  • Perhaps he has his own (minor) variation of the strike zone which teams/batters are very familiar with and that he is consistent with his calls.

Looking at the table, I want to clone Lance Barkdale so he can be the Home Plate Umpire for every game. And if I am a batter, I cringe if Jerry Layne is the Home plate Umpire.

Umpire Balls Called As Strikes Rank Strikes Called As Balls Rank Batter Swung At A Ball Rank
Adam Hamari 31.88% 25 2.16% 7 42.90% 42
Adrian Johnson 35.31% 73 3.68% 63 42.92% 43
Alan Porter 32.65% 37 2.98% 31 41.65% 11
Alfonso Marquez 29.45% 5 4.28% 78 42.13% 19
Andy Fletcher 33.99% 60 4.32% 79 41.70% 14
Angel Hernandez 38.31% 85 2.20% 9 43.79% 67
Ben May 30.67% 17 2.38% 12 44.22% 81
Bill Miller 36.99% 83 2.09% 5 43.31% 57
Bill Welke 32.89% 43 3.51% 54 43.90% 70
Bob Davidson 36.32% 82 3.40% 51 43.78% 66
Brian Gorman 35.59% 75 3.29% 42 43.13% 48
Brian Knight 34.11% 62 3.45% 52 43.32% 58
Brian O’Nora 36.08% 80 3.29% 43 45.17% 87
Bruce Dreckman 36.01% 78 4.24% 76 42.33% 26
CB Bucknor 30.55% 15 3.55% 55 42.16% 21
Chad Fairchild 32.98% 45 2.70% 19 41.95% 17
Chris Conroy 28.98% 3 3.71% 64 43.13% 49
Chris Guccione 30.19% 11 3.00% 32 43.85% 69
Chris Segal 32.43% 33 2.88% 25 43.03% 46
Clint Fagan 30.38% 12 2.64% 18 42.70% 36
Cory Blaser 33.40% 51 2.89% 26 42.86% 41
D.J. Reyburn 33.79% 59 3.37% 48 42.56% 30
Dale Scott 31.22% 20 4.03% 72 42.66% 34
Dan Bellino 32.81% 41 2.92% 28 42.73% 37
Dan Iassogna 37.31% 84 3.57% 56 44.28% 83
Dana DeMuth 29.98% 8 4.52% 83 42.80% 39
David Rackley 35.28% 72 4.25% 77 43.43% 59
Doug Eddings 34.86% 68 2.19% 8 42.33% 25
Ed Hickox 34.38% 66 4.33% 81 40.86% 3
Eric Cooper 33.43% 52 1.74% 2 41.65% 12
Fieldin Culbreth 33.01% 46 3.38% 49 44.16% 79
Gabe Morales 32.70% 38 2.58% 16 44.24% 82
Gary Cederstrom 31.64% 22 3.63% 59 44.59% 84
Gerry Davis 32.36% 32 4.33% 80 43.43% 60
Greg Gibson 29.99% 9 3.71% 65 43.81% 68
Hunter Wendelstedt 33.66% 55 2.86% 24 43.93% 71
James Hoye 29.70% 6 3.85% 67 45.04% 85
Jeff Kellogg 30.12% 10 3.65% 61 40.75% 2
Jeff Nelson 36.02% 79 2.36% 11 44.10% 77
Jerry Layne 35.21% 70 4.61% 86 43.20% 54
Jerry Meals 31.86% 24 3.86% 68 42.68% 35
Jim Joyce 29.84% 7 4.15% 74 45.05% 86
Jim Reynolds 31.23% 21 2.85% 23 42.14% 20
Jim Wolf 33.16% 48 3.14% 37 41.05% 4
Joe West 32.51% 34 4.54% 84 43.20% 55
John Hirschbeck 39.18% 87 2.63% 17 42.56% 29
John Tumpane 30.76% 18 3.23% 39 41.44% 7
Jordan Baker 32.58% 36 3.30% 44 43.00% 44
Kerwin Danley 32.51% 35 3.93% 70 42.34% 27
Lance Barksdale 28.95% 2 2.33% 10 41.12% 5
Lance Barrett 34.38% 67 2.94% 29 43.96% 72
Larry Vanover 33.27% 49 4.09% 73 43.20% 53
Laz Diaz 31.80% 23 3.67% 62 41.83% 15
Manny Gonzalez 35.27% 71 2.74% 20 43.99% 73
Mark Carlson 31.99% 27 3.38% 50 41.49% 8
Mark Ripperger 33.99% 61 2.09% 6 44.19% 80
Mark Wegner 31.94% 26 3.09% 34 42.57% 32
Marty Foster 33.47% 53 3.07% 33 43.75% 64
Marvin Hudson 32.19% 30 3.60% 57 41.63% 10
Mike DiMuro 35.50% 74 3.21% 38 42.56% 31
Mike Estabrook 35.68% 76 2.90% 27 42.82% 40
Mike Everitt 33.79% 58 2.06% 4 42.29% 24
Mike Muchlinski 33.63% 54 3.27% 41 41.89% 16
Mike Winters 32.78% 40 4.17% 75 43.19% 52
Pat Hoberg 34.33% 65 2.54% 14 41.56% 9
Paul Emmel 32.05% 29 3.87% 69 43.07% 47
Paul Nauert 38.63% 86 3.27% 40 42.42% 28
Paul Schrieber 30.39% 13 3.10% 35 44.09% 75
Phil Cuzzi 33.14% 47 1.74% 3 42.61% 33
Quinn Wolcott 33.30% 50 2.81% 22 43.43% 61
Rob Drake 33.70% 56 4.57% 85 41.37% 6
Ron Kulpa 35.74% 77 2.58% 15 43.01% 45
Ryan Blakney 29.20% 4 3.34% 45 43.76% 65
Sam Holbrook 32.95% 44 3.65% 60 42.21% 22
Scott Barry 32.85% 42 2.95% 30 43.67% 63
Sean Barber 32.01% 28 4.46% 82 42.12% 18
Ted Barrett 35.09% 69 3.36% 47 41.70% 13
Tim Timmons 33.75% 57 3.34% 46 42.75% 38
Tim Welke 36.27% 81 3.63% 58 43.47% 62
Toby Basner 30.65% 16 2.48% 13 44.09% 76
Todd Tichenor 32.35% 31 4.01% 71 44.00% 74
Tom Hallion 32.71% 39 4.78% 87 42.21% 23
Tom Woodring 24.33% 1 3.85% 66 40.17% 1
Tony Randazzo 34.14% 63 3.47% 53 43.16% 51
Tripp Gibson 31.03% 19 3.13% 36 43.15% 50
Vic Carapazza 34.26% 64 2.74% 21 43.28% 56
Will Little 30.53% 14 1.65% 1 44.15% 78

A few disclaimers about these results:

  • They are only as good/reliable as the Pitch data in the Game Day data.
  • They depend on Zone being an accurate representation of whether the pitch was a ball or strike.
  • The current technology only measures where the ball crosses the front of the plate. So, for example, pitchers with great curve balls may throw a monkey wrench into any comparison.

And yes, someone in the not too distant future, a comparable post with these metrics calculated for pitchers and batters is planned.

This entry was posted in Analysis. Bookmark the permalink.