Measuring expected save percentage

by James Mirtle on Apr 27, 2009 12:22 AM CDT in Statistics Comment 17 comments

It's no secret I'm a numbers guy. And the one thing I'd love to see more of is more creative and detailed goaltending statistics.

Sites like Behind The Net are starting to make some progress when it comes to measuring how skaters are playing (and who they play with and against), but other than making some educated guesses, we don't generally know which goaltenders are facing the toughest shots and making the most difficult saves.

Gabe Desjardins takes a run at looking at some other stats like expected save percentage and expected goals against average, and while the formatting is a bit rough over on his site, the numbers are pretty interesting. By his count, Tim Thomas improved his even-strength GAA by the most from what it should have been based on the quality of shots he faced, and the next best netminders in this regard were Khabibulin, Luongo, Lundqvist and Vokoun. (You can read about Gabe's methodology here.)

So, which goaltenders faced the most difficult shots at even strength this season? Let's have a look, based on what he's got for expected save percentage (keep in mind that a lower ESPT indicates more difficult shots against):

	NAME	GP	SPCT	GAA	ESPT	EGAA	DELTAGAA
1	HENRIK LUNDQVIST	69	913	2.38	890	3.00	-0.63
2	CRISTOBAL HUET	41	909	2.23	895	2.57	-0.34
3	JOHAN HEDBERG	33	900	2.82	898	2.89	-0.07
4	KARI LEHTONEN	46	917	2.73	900	3.29	-0.56
5	DANY SABOURIN	19	897	2.53	901	2.43	0.10
6	CHRIS OSGOOD	46	894	2.63	901	2.47	0.16
7	N. KHABIBULIN	42	927	1.94	902	2.64	-0.70
8	VESA TOSKALA	53	893	2.85	902	2.61	0.24
9	YANN DANIS	31	915	2.48	903	2.86	-0.37
10	DAN ELLIS	35	911	2.47	903	2.71	-0.23
11	MARTIN GERBER	26	897	3.02	903	2.85	0.17
12	JOEY MACDONALD	49	897	3.32	903	3.10	0.22

Lundqvist facing the most difficult shots in the league meshes with what I've seen this season, as on a lot of nights the Rangers defence has left him out to dry. It's interesting, too, to see Osgood and Toskala show up here, as both had very poor years which could certainly be, in part, the result of seeing a lot of pucks that were very hard to stop.

Anyone surprised to see the Islanders goalies on there?

On the flip side, here's a look at netminders who saw the easiest quality of shots by Desjardins' measure:

	NAME	GP	SPCT	GAA	ESPT	EGAA	DELTAGAA
1	N. BACKSTROM	71	921	2.21	919	2.29	-0.08
2	A. RAYCROFT	31	900	2.74	916	2.30	0.45
3	PETER BUDAJ	56	896	2.74	915	2.25	0.49
4	M-A FLEURY	62	916	2.42	914	2.51	-0.08
5	JONATHAN QUICK	44	917	2.28	914	2.35	-0.07
6	ERIK ERSBERG	28	904	2.41	914	2.17	0.24
7	TOMAS VOKOUN	59	931	2.25	913	2.84	-0.59
8	MIKE SMITH	41	924	2.19	913	2.50	-0.31
9	RYAN MILLER	59	922	2.28	913	2.53	-0.25
10	A. NIITTYMAKI	32	920	2.23	913	2.41	-0.18
11	MARTIN BRODEUR	31	930	1.90	912	2.38	-0.49
12	S. CLEMMENSEN	40	921	2.13	912	2.40	-0.27
13	BRIAN ELLIOTT	31	913	2.38	912	2.39	-0.01
14	J. LABARBERA	28	905	2.58	912	2.39	0.19

Backstrom's not a surprise (and if you need a reason why he shouldn't win the Vezina, there it is) and neither are the two Devils goaltenders. But how about the Avs netminders getting sheltered — or the Kings? Or Mike Smith showing up here despite playing behind a ridiculous number of defencemen?

The key here, however, is outperforming the ESPT figure, which obviously Raycroft and Budaj fall very, very short of.

Now, if we subtract ESPT from SPCT, here are the top five netminders at improving their save percentage at even strength:

Thomas
Luongo
Khabibulin
Lundqvist
Hiller

The five worst by this metric? Lalime, Legace, Budaj, Raycroft and Ersberg.

As I said, it's pretty interesting stuff. It does rely on the scorekeepers in NHL arenas marking where shots are taken from, which can't be an easy task, but it seems what Desjardins is doing is flushing out the best of the best in goal and producing some expected results.

For the results of goaltenders not listed here, see Gabe's original post for all of the data.

0 recs | Comment 17 comments | Add comment | Share

Story-email Email Printer Print

Comments

Not surprised to see the Expected GAA for both Habs goalies to be very high. It certainly did seem like the opposition had a lot of room to move around in their zone and get high quality shots all season.

Also interesting to see that his system saw Jonas Hiller’s ability as early as last season. Very interesting stuff.

by Costa24 on Apr 27, 2009 12:51 AM CDT reply actions 0 recs

Proceed with caution:
- The scorers in some arenas have a systematic bias when reporting shot distance. (This is certainly true of the NHL data, and is probably also true of the ESPN data Gabe uses.)
- Gabe ignores shot type, which is a significant factor in shot quality.
- Gabe ignores shooter, which is also a significant factor in shot quality (Heatley’s shots are much more dangerous than Samuelsson’s).
- All kinds of other factors are currently not even available for data mining, like whether the shot came off a rush, a cycle, a transition, or a broken play. There’s no reason to think any of these things will just “average out” for a specific team.

by RyanV on Apr 27, 2009 2:15 AM CDT reply actions 0 recs

1. ESPN simply uses the same RTSS feed that the NHL provides, they don’t have separate scorers, so the same “bias” problem exists.
2. I don’t have the link handy right now, but there’s little reason to believe that the individual shooter has a large effect. Very, very few consistently over- or under-perform against expectations.
3. There may be other factors that aren’t available like you mention, but it’s a fair bit better than plain old SOG.

More fun than a stick to the face!
On the Forecheck is SB Nation's blog covering the Nashville Predators.

by Dirk Hoag on Apr 27, 2009 9:01 AM CDT up reply actions 0 recs

One other note about “rink bias”. To my knowledge nobody has attempted to audit the results from different rinks (which would admittedly take a lot of time and effort), but the bias is assumed because average shot distances are shorter in some rinks, and longer in others. It is possible that team play has something to do with that, but we really don’t know to what extent without some form of independent audit.

More fun than a stick to the face!
On the Forecheck is SB Nation's blog covering the Nashville Predators.

by Dirk Hoag on Apr 27, 2009 9:24 AM CDT up reply actions 0 recs

@forechecker – Is there a site out there that describes, generally, the basic differences between rinks in each arena? Just curious really.

'Nucks Misconduct - Housing Swedish Millionaires Since 2000.

by Yankee Canuck on Apr 27, 2009 9:46 AM CDT up reply actions 0 recs

Go to HockeyAnalytics.com, and look for an article titled “Product Recall on Shot Quality.”

More fun than a stick to the face!
On the Forecheck is SB Nation's blog covering the Nashville Predators.

by Dirk Hoag on Apr 27, 2009 10:04 AM CDT up reply actions 0 recs

My analysis showed there were 2 teams with significant differences in their home ice shot distances (consistently over several seasons):
NYR: about 7’ closer than should be reported (reported as 21’ instead of 28’)
StLouis: about 3’ closer than should be reported. (reported as 25’ instead of 28’)
Other arena’s showed some error in some years and seemed corrected in others so it’s hard to tell with the other teams, but the other arenas are only off by a foot or so. My statistics are adjusted accordingly… These adjustment still leave Ludqvist as a good goalie, but not a “superhuman” goalie….

by hockeynumbers on Apr 27, 2009 12:47 PM CDT up reply actions 0 recs

2. Actually, differences in shots between individual shooters is a large effect. Differences above and below career averages are nothing to be concerned about, but different players will have very different (and predictably different) career averages. Some of it has to do with physical shooting ability (Heatley has a harder and more accurate shot), but most of it has to do with shot selection (Heatley is a finisher who takes few shots overall, but many of them are one-timers).
3. Yeah, the shot quality data is an improvement over raw shot data. I did say “proceed”… I just wanted to add “with caution” and point out some of the flaws that weren’t mentioned in the original post. Stats are useless if you don’t understand their shortcomings.

by RyanV on Apr 28, 2009 2:39 PM CDT up reply actions 0 recs

Yeah I think Dirk has the right idea … think of this as just a slight upgrade on save percentage that weights the value of shots closer to the goal. It’s still useful — just not the be all and end all of goalie stats.

Check out my blog at fromtherink.com

by James Mirtle on Apr 27, 2009 11:34 AM CDT up reply actions 0 recs

A couple notes/surprises

Of concern is that the ESPT is that the distribution of the playoff goalies is all over the place. You would expect playoff goalies to, in general, have higher expected save percentages. That’s not the case, as playoff goalies are distributed pretty evenly throughout the ESPT distribution. There are 12 goalies from teams in the playoffs in the top half of ESPT and 15 in the bottom half, and only 1 goalie in the top 9 in ESPT made the playoffs. Does this mean that in order to make the playoffs the key is to have strong goaltending and weak defense?

If you look at goalies on the same team, it seams that the “expected Save%” can vary considerable when judged by those of their teammates.

Jose Theodore: ESPT – 906, Brent Johnson ESPT – 910 (not much varience)
JS Giguere: ESPT – .910, Jonas Hiller – .906
Dan Ellis – .903 Pekka Rinne – .907
Ty Conklin – .908 Chris Osgood-.901
Smith – .913 Ramo – .907
Anderson – .909 Vokoun – .913
Price – .909 Halak – .904

Certainly some of these fall within the scope of statistical variance (most a few 1/100ths of a percent), but 5-10 100ths of a percent is seen as a pretty big flux it’s a big deal.

I’m actually extremely surprised by this. I thought, pretty much all season, that Raycroft and Budaj were victims of better shot quality owning to poor defense.

In fact the Avs defense was just miserable this season, extremely porous. I find it incredibly hard to believe that Avs goalies, collectively, faced the easiest shots. One explanation for this could be: The Avs defenders block a lot of shots (in fact that’s what their defense is based upon), and It’s always been my partial hypothesis that the shots that are blocked are the easiest to stop, meaning that the ones that get through are of higher quality.

Also it’s possible that with emphasis on getting in the shooting lanes to block shots has the counter-productive effect of screening the goalie, making normally easier shots much more difficult.

Those are just some initial thoughts on this stat. I think it’s an okay beginning, but I think the faulty assumption is that shot quality is only a function of distance from the goal. While it’s certainly a factor I think there’s a lot of other factors that go into a shots quality as well that aren’t taken into account.

The 2008-2009 Colorado Avalanche: Slumpbusters

by Jibblescribbits on Apr 27, 2009 11:52 AM CDT reply actions 0 recs

Agreed that the Avs showing up there is a surprise. But, on average, shot distance and location is a huge factor (see Gabe’s chart here).

Those differences between goalies on the same team aren’t that hard to explain. Most of those netminders played a far different number of games, and goalies with under 35 games played would have such few minutes played (at 5-on-5) that it’ll skew things.

It’s interesting, too, if you look at goal support, something goalies have no control over, you’ll see the same thing. Often, teams score more goals over the course of the season with one goalie in as opposed to the other. Make sense of that.

Check out my blog at fromtherink.com

by James Mirtle on Apr 27, 2009 12:11 PM CDT up reply actions 0 recs

Oh

Of course shot location are factors in shot quality, but there are other factors that go into it as well. A team that allows opposing players to pass well within their own zone will generate higher quality shots from further out than a solid defensive team that gets their sticks in the lane. I’d imagine the quality bubble on Gabe’s chart for the Avs would be bigger than the one he used… The question on that would become: Is that because of crappy goaltending, or crappy defense. I think the obvious answer is: a lot of both.

The more I think about it, the more is seems like the discrepency for goalies on the same team probably falls within a standard deviation, especially when small sample sizes are taken into account.

It’s interesting, too, if you look at goal support, something goalies have no control over, you’ll see the same thing. Often, teams score more goals over the course of the season with one goalie in as opposed to the other. Make sense of that.

The team just feels more confident in front of one goalie. they play more confidently because they know their goalie will bail them out!

/nonsense spouted from Avs fans when Raycroft was 9-1 earlier this season.

The 2008-2009 Colorado Avalanche: Slumpbusters

by Jibblescribbits on Apr 27, 2009 1:44 PM CDT up reply actions 0 recs

I’m surprised not to see Steve Mason in the second list. A lot of critics said he was just a product of the Jackets’ system, much like LeClaire before him. According to these numbers, he’s not.

Also strange to see Brian Elliott on that list, but I suppose that’s because he took over when the Senators started playing well, rather than being there during the rough times like Auld.

by DarrenM on Apr 27, 2009 12:14 PM CDT reply actions 0 recs

Raycroft

sucks.

Sports And The City

A Toronto sports blog, where we unequivocally and unapologetically support the home team, and Mats Sundin, the GREATEST TORONTO MAPLE LEAF EVER...

PLAYOFFS!!!!1

by eyebleaf on Apr 27, 2009 12:32 PM CDT reply actions 0 recs

Yeah, I’ve gotta admit… all of this algebraic stat crunching is why I’m definitely not a “numbers guy.”

This work from Behind The Net is certainly wonderful for those who have longed to look beyond GAA and Save Percentage, but I still prefer firsthand evidence.

"Sweet Zarley Zalapski, Batman!"

by ZarleyZalapski on Apr 27, 2009 2:04 PM CDT reply actions 0 recs

what do you consider “firsthand evidence”? The method I’ve proposed here relies on first-hand observations made by the scorers in various arenas (shot distance and angle) and uses no complex math (just +, -, *, /) It’s not substantially different than watching tape and counting scoring opportunities, which is why several NHL executives use the method I’ve described to evaluate goalies.

by Hawerchuk on Apr 27, 2009 4:06 PM CDT up reply actions 0 recs