There was an interesting exchange over at Battle of Alberta today:
Cosh: Is there actual evidence that there is any point in breaking overall save percentages into smaller, noisier samples? Or is this just cargo-cult sabermetrics (it works in baseball, so…)?
After some further discussion that included, of course, a reference to Roy Halladay’s fine record against the Yankees this year, Fenwick kicked out the following:
Matt: In 06/07, two goalies had >.900 PKSV%. Their overall save percentages, between that season and this past one, dropped 27 and 9 points. Is that the same as parsing this fellow Halladay’s record for little slices that aren’t likely to be repeated? I don’t think it is, but again, I don’t know what it is you want to see.
This intrigued me, for a number of reasons. First of all, because it suggests a simple rule, a line in the sand. “Goalies who have big PPSV% seasons can expect their overall save percentage to fall in the following season.” We can’t call this a Fenwick number, for obvious reasons, but the principle needs something to go by and doctrines seem popular at the moment, so the Lethbridge Doctrine it is.
How does the Lethbridge Doctrine hold up? Fairly well, I think. Before I go to my results, let me explain how I did it. As everyone knows, the NHL save percentage has moved over time. This is the function of a couple of things: first, generally, fewer pucks have been going into the net and, more subtly, the percentage of shots that are taken on the PP, which are much more likely to go in, has jumped around a lot, from a low of 17.4% in 1999-00 to a high of 24.8% in 2005-06. The goaltending data from 1998-99 to the present date looks like this:
So as to have a workable set of data that isn’t swayed by shifting save percentages, I scaled everything to a league in which the average save percentage was .907 and in which the splits were .917/.866/.916. I’ll call this a scaled save percentage. So as to limit the extent to which the data is altered by shifting proportions of ES/PP/PK shots, I also calculated save percentages for all of the goalies in this little study as if they saw 76.1% ES shots, 20.0% PP shots and 3.9% SH shots.
Next step was to define a group to look at. I just picked the top five from each year in PPSV% (minimum 200 PP shots faced) and compared their scaled save percentage from that season to their scaled save percentage the following season. The results are pretty interesting.
Of the forty goalie seasons that gave me, 33 of them put up worse scaled save percentages in the following season. The average decline was by 0.010 worth of save percentage. Every hundred shots, that’s a goal that wouldn’t have been given up the year before. That’s effectively an extra goal against every three or four games. Two stayed the same and five got better. 25 of them were at least 0.008 worse, which is a big dip.
What does it all mean? Well, based on the last eight years, you might reasonably argue that four of the five best PPSV% goalies in the NHL will suffer drops in their scaled save percentage this season. Who will be the five staring down this unfortunate history in the coming season? Martin Brodeur, Niklas Backstrom, Carey Price, Tomas Vokoun and…Mathieu Garon.
Garon actually carries a special burden into the coming season - he had the best PPSV% in the NHL in 2006-072007-08. Of the eight players to lead the NHL in PPSV% since 1998-99 who have already played their next season, the average decline in scaled save percentage was 0.017, although it’s kind of odd - Ed Belfour (02-03 PPSV% leader) and Henrik Lundqvist (05-06) had declines of .006 in scaled save percentage, Roberto Luongo (03-04) and Martin Brodeur (99-00) had declines of .008, Dominik Hasek (98-99) had a decline of .015, Jose Theodore (01-02) had a decline of .026, Chris Mason (06-07) had a decline of .032 and Manny Fernandez (00-01) had a decline of .033.
The better places on that list have some starrier names. If I was a betting man - and I’m not - my guess for Garon would be pretty close to league average, as opposed to last year, whereas his scaled save percentage for last year was .914. I have a hard time seeing how the Oilers make the playoffs with that, no matter what that knob/idiot Mirtle says.
Garon, an excellent save percentage on shootouts two years running, and an excellent save percentage on powerplays two years running.
Proof that Garon was lucky or that he is a clutch player, who can raise his game when the going gets tough?
Whatever the answer is — I have no strong opinion on it, though your numbers indicate luck is a big factor here — I agree with all those who write the crucial question for the Oilers is the one that isn’t being asked much right now — How will Garon perform?
Me, I think he’s going to be strong, in that he won’t let in many weak goals this year.
By my (controversial) count, he made just 24 miscues that directly contributed to goals against last season, while Roloson (who was horrendous most of the year) made 41.
Per even strength minutes, that broke down to one such miscue for Garon every 85 minutes, and one every 44 minutes for Roloson.
Will Garon let in more softies this year, the kind of goals against that really break a team’s back? From what I saw last year, he looked solid enough and had a few stretches were he was all unbeatable. If he can do that again, the Oil are in the playoffs. If he can’t, as you suggest Tyler, they are out.
Why is Mirtle an idiot, or was that said in jest?
With apologies to Mirtle, I’m focused on the Oilers, so I rarely read his blog. But from what I’ve seen, he’s not exactly controversial, or out there in his views.
Garon, an excellent save percentage on shootouts two years running, and an excellent save percentage on powerplays two years running.
His PPSV% was below league average in 2005-06 and 2006-07. His shootout save percentage appears to have been below league average in 2006-07 as well. With that said, I think that there is a reasonable argument on the evidence available that he’s a good shootout goalie. Going into last year though, his career average in PPSV% was .840. He put up a .911. So I don’t think that your point has any real basis.
My overall take on Garon is that he’s a guy who is a great steal at $1MM or so. I don’t want the Oilers to lock into paying him $3MM a season for two or three years.
David - he posted his predictions and a few hours later had a post up about how he didn’t appreciate being called a knob and an idiot. I’ve gone out with him for drinks a couple of times - he’s a good guy who knows a ton about hockey.
Garon actually carries a special burden into the coming season - he had the best PPSV% in the NHL in 2006-07.
That should read “in 2007-08″, which is why Staples wrote what he did in #1. As you note in #3, he was actually below average in 06/07.
And personally, I think Mirtle is a lot more knob than idiot.
Do I understand you to say that the average regression to the mean (vs. league) of a selected group that was selected from the best goalies to begin with (and would therefore be expected to decline away) was 0.01?
That should read “decline anyway.”
Har har har.
Is it not possible that the Oilers scores quite a few more goals this time around and that offsets the drop off in goaltending? I think they’re definitely in that 6-10 mix in the West.
(and would therefore be expected to decline away)
Jeez, who’s the cargo-cult sabermatrician now? Wasn’t this very point what you were up in arms about yesterday?
I don’t see how Tyler could have been much clearer. What is it that you don’t believe? Or are you just trying to argue that “effectively an extra goal against every three or four games” is meaningless? (Random? what?)
I’m not trying to argue anything, cranky pants, I was asking whether I’d interpreted Tyler’s finding correctly. Just for starters, I’d like the super-clear to be made super-duper-double-clear by having somebody tell me whether “the average decline was 0.01″ means that only the 33 decliners in the group of 40 were included, or whether -0.01 is the delta for the whole group.
If I have understood it, this seems to be an airtight proof that Garon can be expected to lose a point off his save percentage next year, more often than not. If that’s the case, and setting aside my instinctive reaction (”Big fucking whoop”), I’d be curious to know whether this is larger than the overall decline that could be expected amongst goalies in the same general class whose PK-EV-PP statistics aren’t unusually warped. It’s a question, not an assertion. Is it OK to wonder whether a trait observed among green elephants is true of elephants generally?
P.S. Kipper sux
Does Duhatschek know that that you and James dated??:)
I guess it must be at least semi-serious if Mirtle took time out of his busy schedule running around the ACC Centre.
Luck or randomness seems to be a fairly strong component to PP performance on the offensive side of things. If you look at team PP% the correlation from one to the other is not that strong. Same with individual level data for players. The correlation is not that strong from one season to another. It makes sense to me that the same would be true for goaltenders as well.
Is it that PP performance is simply more random because of small sample sizes provide poor measurement–or that luck is simply more important than at ES?
I tend to favor the 1st explanation. In theory at least a team has MORE control over the puck with the extra man and in theory you’d expect them to be more consistent since they have possession more consistent than at ES. But when PP TOI is just a fraction of ES TOI, the statistical weight of a fluky goal or lucky bounce is exaggerated that much more. So it seems to me it is a measurement problem.
If it is a measurement problem, I wonder if a goalie’s PP SV% could be forecast more accurately by using a weighted average of the TWO previous seasons? This is certainly true for forecasting skater PP scoring rates. A two year average improves the R-squared substantially IIRC.