Mirtle has a post up dealing with this year’s +/- leaders and laggards, along with some information about their on-ice EV SV%. It’s in the vein of the PDO numbers stuff that I talked about a few days back.
One of his commenters has chimed in with the following:
So now it’s the goalie’s fault the guys in front of him have bad +/-.
Phhht.Doesn’t Brindamour play in front of the same goalie that Eric Staal (+8) does?
The next worst +/- on the Hurricanes is Whitney at -10. In total, 13 Hurricanes are -1 or better. Maybe Brindamour can’t pivot because of the knee surgery. Either shut it down or retire already.Kris Draper is what, 58 years old? Maybe age has more to do with his horrible +/- than what Osgood and Conkin are doing. Twelve Red Wings are -1 or better.
Boyes, Bertuzzi, McDonald. All horrible lifetime minuses. Nobody would ever confuse these players with guys who backcheck. Even Turk Broda couldn’t rehabilitate their +/-.
The +/- might not be perfect, but all you have to do is take a glance at a team to thumbnail it a little better. If a guy is double-digit minus on a team of pluses, he blows.
No need to twist it into GF+/- divided by GA+/- X uniform number divided by GAA X Sv percentage.
I don’t necessarily think that it’s the goalie’s fault that the fellows Mirtle refers to have poor +/- but I think that that’s the point he’s making either. I think that what he’s really getting at is that there’s some randomness at play here. At this point of the season, lots of guys have been on the ice for somewhere between 150 and 250 shots against - a very small sample to be sure. We know, from observing goalies, that 200 shots doesn’t tell us very much about a goaltender. Martin Brodeur had a 200 shot stretch in which his ES save percentage was .965 last year; he had a 200 shot stretch in which his ES save percentage was .890. Roberto Luongo had a 200 shot stretch in which his ES save percentage was .980; he had a 200 shot stretch in which his ES save percentage was .885.
The essential dispute is whether those stretches tell us anything at all. It’s sort of a dispute betwen randomness (also referred to as “luck” amongst the staterati) and a hot streak or a guy playing really well. Personally, I’m on the side of the randomness fence - no matter what those guys do, they aren’t going to be .980 ES save percentage goalies in the long haul. Maybe some pucks hit the post, maybe some guys on the other team flubbed their chances; who knows. I don’t really worry that much about it, because I don’t think that you can tell anything from it.
Turning back to the real subject of Mirtle’s post, save percentages behind players, I should mention a quick caveat - Mirtle is citing ES save percentages that he’s found on Desjardins’ site. Gabe doesn’t, as far as I know, strip the ENG. Carolina’s given up 4 ENG this year. Vic Ferrari has Brind’Amour at 25 ESGA, with an .882 on-ice save percentage; Gabe has him at 26 GA with an .853 on-ice save percentage at the moment. Vic’s counting ES, which I assume includes 4 on 4 and Gabe isn’t; I assume that that explains the rest of the discrepancy.
The thing of it is, despite what Mirtle’s commenter is saying about Draper being old and Brind’Amour being unable to pivot because of knee surgery, guys with great ES SV% at the halfway mark fell down by a lot in the second half; guys with terrible ES SV% tended to bounce back. Here’s the top 50 players in the league in terms of on ice save percentage* through the first half last year, along with their second half save percentages. Pay attention to the totals line at the bottom:
.944 through the first half of the season and a perfectly league average .920 thereafter. Here are the worst fifty*, for both the first and second half. Again, pay attention to the totals line at the bottom:
An atrocious .881 through the first half; a hair off league average at .918 thereafter. Did they all make massive improvements? I can’t prove that it didn’t happen but I find it exceedingly hard to believe.
Two groups of players with completely different first halves and virtually identical second halves. I don’t know how anyone can seriously criticize Mirtle’s comments on this topic. I’ve made my views clear that I think that the save percentage stuff is largely randomness at the NHL level. Time will tell but if there’s some sort of a market in which you can make bets based on the percentages behind guys through the first half of the year, I think that we’re starting to pile up the evidence that the smart is against betting on players continuing to post big save percentages over time.
Just as a final aside - I’m sure it’s not news to anyone here but Mirtle’s commenter misses the point citing Bertuzzi, Boyes and McDonald. For one, Bertuzzi and Boyes posted plusses last year. More importantly, there’s more to +/- than just save percentage; the shot ratio goes into it as well as the save percentage. McDonald is currently -12 in 16 games; Bertuzzi is -11 in 26 games and Boyes is -14 in 25 games. You only need to look at their history to see how out of line these +/- numbers and to realize that there must be something out of the ordinary driving the short term results.
*Includes only players who stayed on the same team, kept the same number, were listed at a consistent position in the NHL.com game files and were on the ice for at least 200 ES events in each half.
Everything looks pretty conclusive from where I stand, Tyler.
Of course, there’s going to be a ton of folks who just chose to disagree with you.
Damn, Tyler, you and Matt have just been on fire lately. Terrific stuff again.
I’ll have to invest some time to make this data available for back to 02/03. I have a script that generates that from the play by play and shift charts for the early years.
There are quite a few games with missing shift charts or missing play-by-play, esp in 02/03. Still the missing games aren random so it shouldn’t have an effect on the kind of thing you’re doing here.
The script I have to do that, it literally would take several days to run one season. It is written in VBA and goes player by player individually. I’ll have to rewrite that in php, and in a better way, and start using MySQL for the current stuff and future stuff on my stats site, just to speed things up and to offer more flexibility.
And I’ll have to check the TOI vs shiftchart vs play-by-play (current and old format) to make sure that the accuracy is there. Fortunately there are periods of overlap TOI and shift charts, so I can use that to double check as well.
All in due time.
I remember some wacky kid from HF screaming that individual defensemen were having a profound effect on the EVsave% behind them. So I asked him to list the 20 guys he thought were the real difference makers. So he listed guys that had huge +/-’s that year (03/04), but I never checked for that year, I was tricky and used (02/03) instead, and the net effect of these guys on the EVsave% behind them was +0.0015. So when they were on the ice one extra shot in 620 was stopped by their goalies. Not the kind of thing a guy should be able to notice with the naked eye.
Do you guys come out with a SV% differential?
Great stuff as usual Tyler. I am definitely convinced by the theory that there is a lot of randomness at play. One question that still remains for me and a lot of folks that discount that randomess theory entirely is the effect of shot quality. Who limits it and who doesn’t? Or is the gap between good and bad so small at the NHL level that guys can’t really do that? It’s intuitive for me that if I’m playing defence for the Oilers, there are going to be a ton of chances against and I’d expect it to show up in my on-ice save percentage numbers. Should the same thing be expected of NHL rookies? Of call-ups from the minors? Of Sebastien Bisaillon coming straight out of junior? These are the things that I’m struggling with at this point.
Or is the gap between good and bad so small at the NHL level that guys can’t really do that? It’s intuitive for me that if I’m playing defence for the Oilers, there are going to be a ton of chances against and I’d expect it to show up in my on-ice save percentage numbers.
I almost wrote pretty much this, about how if played D for the Oil, I’m sure that my on-ce save percentage would be dismal. There’s something called DIPS theory in baseball that I think might have some application in this area.
Should the same thing be expected of NHL rookies? Of call-ups from the minors? Of Sebastien Bisaillon coming straight out of junior? These are the things that I’m struggling with at this point.
Interesting idea. That’s something that I can check out, whether as a group, bottom end guys do worse. I’m open to believing that individual players have an effect on save percentage, I just tend to think that it’s so small that it can’t be seen for the noise. Maybe if we group them together, it’ll be easier to identify. Good idea.
vic, your scripts are awesome. is there any place where i can find a list of all the ones you’ve written (and that currently work)?
tyler,
this is a serious question.
how big an effect on ES SV% do you think goaltenders have?
or more accurately, how big a difference in actual skill is there between them?
Tyler:
Yeah, I agree with your last post, there is definitely an effect, it`s just really small and hard to see.
I think it is very significant when it comes to PKing though.
We`re going to need to engage Ken Krzywicki and Jlikens to crack the PKing nut, methinks. The former has modelled shot quality in a manner that has genuine repeatability, and will certainly predict future EVshooting% much better than EVshooting% itself.
Unfortunately he didn`t break it down by game situation (EV, 5v4PK, 5v4PP, etc.). And he seems to have abandoned it two years ago. I suspect that once he realized that home ice recording bias was giving the stat most of it`s significance it didn`t seem worth doing. The impact is very small on the whole, especially for the effect on save%. And this before accounting for the shooting% effect of schedule and randomness.
But there is an undeniable truth in his methodology. And without having the raw data I can`t say for sure, but with a super-crude check of Ken`s overall result by team vs PKsave% and EVsave% correlations, well the PK stuff is driving the bus by a wide margin. And it is of course, the much smaller sample, so just using the PK stuff should may give us something
really big.
This
http://objectivenhl.blogspot.com/
is a must-read on the subject, btw. It is rare that someone so good at math has such a reasonable grasp on our game of hockey. Huge props to Jlikens.
I`ve been meaning to post on the subject, and I`ve been suffering through a brutal flu this past week, so I`ve been especially prolific and especially surly this week. But that will end shortly, I`m feeling pretty good today. I hope you or Matt take on the subject though, it`s well worth it imo.
Sunny
There is a whack of stuff on there that I can`t even remember what it does, other stuff is unsafe (and I get hacking attempts quite a bit, apparently I piss off computer guys a lot).
I have some other stuff on there that needs a bit of explaining, and the most important thing, of course, is to engage the guys who know hockey and have clear heads. And for whatever reason these guys are typically really shitty at math (not that I`m one to talk).
So we need to show them how simple the math is, at least the math that a guy like Tyler is using. How it`s rooted in reality and simple observations, and that it`s really piss simple. Of course everthing is simple after you know how to it, and confounding when you don`t.
Then I think we need these 100 or so people, and we need a better place to talk hockey than here.
All in due time.
The two charts are a very good way of making this point.
Vic, if you pissed off “computer guys” a lot, you wouldn’t notice. You’d just be hacked. I’d guess you’re just suffering the pains of having a website.
If the math is simple, why don’t you just explain it? Here, there, anywhere.
MikeP
I will. Not that I’m the most knowledgable person on the subject, or a natural teacher either, but you can be guaranteed that I will keep it simple.
On a skim through a thread at LT’s the other day, Showerhead mentioned that he didn’t know what Poisson or Gaussian distributions were. And I realized that we’ve gotten to the point where a lot of us blast through some of this stuff without full explanation. Or more commonly, commenters bring in this math, generally inapproriately, and make the subject seem more complicated than it really is.
Also Mike
The reason that I thought people were trying to hack into timeonice was that I’ve seen series of hits where posix expressions are used instead of the ‘gamenumber’, ‘first’, etc.
A google got me thinking that probably is was people up to no good. Now obviously there is nothing of value on that site, everything there is just a presentation of the NHL’s data in a different way. So I assume that it’s just mischief, or someone wanting to delete the scripts there. I dunno. Am I missing something?
Tyler:
As you seem to have the data to hand.
If you take the (slightly false) notion that every player should be expected to have EXACTLY the same EVsave% behind him as everyone else on the team (you’ve shown convincingly here that this is very close to true, you would struggle to find people to bet against you on future results, I’m sure).
Then the “effect of EVsave% on EV- of individual players” should be perfectly random.
So a simulation of players EV goals against, where shots is the number of coin flips and the team EVsave% for the year is the weighting of the coin, that will yield an expected distribution of results.
If you do that a bunch of times and take the average it would be best.
Then display that and the actual results of all NHL players in graphical form … it will be two bell shaped curves, almost on top of each other. You could wiki up some tests for normalcy, but it will be off the hook in both cases.
Then (and hopefully JLikens corrects me if I have this wrong), you subtract the variance of the coin-flippers (expected spread of results if individual players have no effect, relative to other NHLers, on EVsave%) from the actual with players (observed spread of results), which should have an ever so slightly larger variance.
Take this variance difference, take the square root of it (std deviation), divide it by the std deviation of the observed … and voila, that yields the % contribution of individual players to EVsave%.
And of course it will be very small in comparison to randomness, because if it was significant, we’d be able to predict which players would have good on-ice EVsave%’s in the future with some success … and nobody seems to be able to do that.
I think that methodology is correct, though I’m far too lazy to even dig up my old stats texts, much less read them. I await Likens’ take on that.
I’ve been meaning to do that myself, and probably will eventually, but the alternate strategy (doing nothing and waiting for you or Matt to take a run at things) seems to be working pretty well for me lately.
Since the question of players’ effects on EVsave% is pretty much answered (though this exercise would put a nice exclamation point on it), I’m more interested in seeing how much of an effect players have on EVshooting%. Then I’d like to know how big this effect is compared to outshooting.
Just to put it on the record, I’m guessing league wide it’s about one third as important as shots for and one sixth as important as outshooting.
@Vic - Two more weeks till Christmas, two more weeks till Christmas. I’ve got all sorts of crazy ideas for shit to do when I’ve got a little time; that one’s not half bad.
@JeffJ - See comment to Vic.
mc/jeff:
Yeah, as jeff says it probably does little more than hammer the point home at this point. Still, would be nice to put a number on it, even if we already know that it will be very small.
With the model built though, we could check the save% effect of individual PKers. I suspect that’s a significant number, though it’s just my sense of it.
And of course the effect of individuals on shooting%, which of course is going to be big. Still, the randomness is probably a big chunk of any one season’s results for any one player.
Vic -
I might try to dig into the shooting percentage tonight. As I mentioned in one of the threads discussing this, it’s interesting because it has a much higher correlation than save percentage, particularly for forwards, which makes sense.
I’ll have to pull out the PK numbers. Over Christmas maybe.
I appreciate the compliments Vic, but I feel compelled to admit that I’m no statistician; I’m simply a fan of the game who prefers to view it in quantitative terms.
Your proposed method of determining the effect of individual players on EV save % against seems correct on an intuitive level. However, I can’t confirm that it is, in fact, correct.
While Ken Krzywicki may have stopped publishing annual studies on shot quality, his associate Alan Ryder still does it. Though, for whatever reason, last year’s study only contained data on shot quality against. This is strange considering that SQF is at least as valid as SQA, and I’m not sure what his reasons were for doing this.
Nick Lidstrom’s On Ice SV% for the last 8 Quarters
Time-St.Nick–Team–Over/under perform Team
08Q1 .963 .923 +
08Q2 .921 .936 +
08Q3 .929 .922 +
08Q4 .925 .919 +
09Q1 .918 .900 +
09Q2 .926 .923 +
09Q4 .925 .921 +
09Q4 .934 .901 +
I know that even patterns can appear in random numbers. But the eyeball side of my brain has a hard to accepting the idea that it where it is Nick Lidstrom or Derek Meech on the ice has zero effect on the ES SV%.
In fact, Derek Meech’s numbers scream “minor league” to me.
Time-Meech–Team–Over/under perform Team
08Q1 .875 .923 -
08Q2 .846 .936 -
08Q3 .947 .922 +
08Q4 .923 .919 +
09Q1 .864 .900 -
09Q2 .893 .923 -
09Q4 .896 .921 -
09Q4 .805 .901 -