mc79hockey.com

Wow. That’s a hell of a post. The thing that really bothers me about it is what this means for SV%- I’ve always thought that it’s just about the best metric we have for goaltenders, but if the edge for the outshooting team is so small, how valuable is it really?

Geez, I forgot how snippy I got in that thread. There were a few threads around that time that turned into bench brawls.

I would definitely be interested in seeing where that analysis of consistently good teams (i.e. Detroit), consistently bad teams (i.e. Columbus), and the inconsistent teams (i.e. Edmonton) lands. Probably about what you’d expect, but it would certainly be good to have a nice solid cushion of evidence to fall back on, however it turns out.

Tyler: I had hoped you would return to this subject eventually, and thanks for doing so in spades with this new information. I think it’s critically important that if we are going to use shots for and against as an important measuring stick for players, that we have a clear understanding on the relationship between shots and results, for both the league and the locals.

As that at-times-heated debate raged, I did my own study on results of outshooting/outshot by, and submitted a post which unfortunately never got published due to some sort of tangle in the ‘Net (a coding or formatting error by me, most likely). Let me try again, as much of that rather lengthy comment is germane to the renewed discussion. I’ll leave out the rancourous bits and edit the rest.

At the time NHL.com stats only went back to 2000 – you’re right they’ve added a whole shitload of stuff in the last few weeks, and some of it is damn useful. But I can certainly corroborate your results on the bottom portion of your big table.

I haven’t looked at team specific cases, which will be topic of my next post on this subject.

I have looked in depth at one team, the Oilers, an odd team which has never really conformed to an outshoot-to-win strategy, to put it mildly. While I follow the league as a whole, I have studied our local team pretty intently for the last 36 years, from which my default position is that (quantity of) shots on goal are simply not that important. From strictly an Oilers’ perspective, I’m disinclined to accept that Corsi numbers are the way to judge individual success, because outshooting certainly hasn’t driven team success over the years.

For starters, I have finally been able to follow up on my suspicions of the 1980s Oilers. Thanks to Hockey-Reference.com for not only publishing the information but responding to a reader suggestion to provide team totals. It turns out the Gretzky Oilers were collectively outshot by their opposition in four out of four regular seasons that they won the Stanley Cup, and 8 of 9 overall (the exception being 1985-86). Interestingly, the Oilers outshot their opponents in four out of four playoff seasons that they went all the way, confirming another suspicion of mine that while the dynastic Oilers were content to trade chances during the season, they dialled it up a notch in the playoffs, especially defensively. But they were good enough in those regular seasons to win three President’s Trophies or equivalent despite a (small) deficit in gross shots on goal. They had the better goalie most games, and the better snipers every game. They didn’t need to outshoot to outscore, that’s for sure.

The 1990 champs had a slightly different formula for success, as they were outshot in both the regular season and the playoffs. No surprise that was the one year their goalie won the Smythe.

More recently, as I detailed in the Horcoff thread, the Oilers succeeded in the 2006 playoffs while being largely outshot, and again down the stretch in 2007-08.

But it’s way more than 20 games here or there. Here’s the Oilers’ record in games outshooting/outshot by their opponent this year:
Outshooting: 25 GP, 11-13-1, 23 points, .460
Outshot: 56 GP, 30-21-5, 65 points, .580

Shots Even: 1 GP, 0-1-0, 0 points, .000
Total: 82 GP, 41-35-6, 88 points, .537

That’s quite a difference, .580 to .460, especially given the variants theoretically weight the coin on the other side. So am I being a contrarian to question the theory?

That’s also quite a difference, 65 points to 23. 65 is a hell of a lot of points from games Oilers were supposedly outplayed. I will grant you a bunch of them came from the shootout but whatever, them’s the rules, and a tie with a chance at a second point is a good result in a game where you are outshot, no? Oilers generated points from 35 of those 56 games (63%), but from just 12 of 25 (48%) of the outshooting games. Whatever the root cause, surely we can agree that this is odd. Or is it all just luck??

In 2006-07 the circumstances surrounding the team were more grim, but once again they displayed an inability to win games in which they held the edge in shots.
Outshooting: 26 GP, 10-15-1, 21 points, .404
Outshot: 50 GP, 20-24-6, 46 points, .460
Shots Even: 6 GP, 2-4-0, 4 points .333
Total: 82 GP, 32-43-7, 71 points, .433

Look at that: points from 42% of the games we outshot the opposition, and from 52% of the games the bad guys held the “edge in play”.

While the measuring stick is crude (outshooting 27-26 is the same as outshooting 49-23) it becomes more meaningful as the sample size becomes significant. We’re now up to a large sample of two seasons, and the Oilers have won or “tied” 45% of the games they’ve held the shots advantage, and 58% of the games that they haven’t. Is that still just luck? Distribution smear? Or do the Oilers play differently with the lead?

As it happens NHL.com ha(d) these statistics dating back to 2000, corresponding with the beginning of Craig MacTavish’s tenure behind the Oilers’ bench. Over those years the Oilers have outshot their opposition by 15,817 to 15,789, a difference of 28 shots over 574 games, or 0.05 shots per game. They have outshot the opposition 273 times and been outshot 277 times. Call it even. Here’s MacT’s career record by this metric:

Season — GP / Pts% outshooting | outshot
——————————————————————-
2000-01 – 39 GP, 0.564 | 40 GP, 0.563
2001-02 – 46 GP, 0.554 | 32 GP, 0.563
2002-03 – 40 GP, 0.550 | 39 GP, 0.564
2003-04 – 41 GP, 0.524 | 36 GP, 0.583
2005-06 – 56 GP, 0.589 | 24 GP, 0.563
2006-07 – 26 GP, 0.404 | 50 GP, 0.460
2007-08 – 25 GP, 0.460 | 56 GP, 0.580

Outshooting: 273 GP, 125-106-24-18, 292 points, .535
Outshot: 277 GP, 129-101-19-28, 305 points, .551
Shots Even: 24 GP, 9-10-4-1, 23 points, .479
Total: 574 GP, 263-217-47-47, 620 points, .540

There just doesn’t seem to be anything at all there supporting success through outshooting. 5 of the 7 years the Oilers had a better record when being outshot, in another the difference was a negligible .001, and even in the 2005-06 (regular) season when the Pronger Oilers controlled a large number of games by the bunches, they won 28 of 56 games they outshot their opponent, and 13 of 26 in which they got outshot. This was the exactly .500 Oilers who went 28-28 in regulation and 13-13 in three-point games; turns out they had an identical split in games outshooting/outshot by.

So now let’s step back and look at this metric across the league. Now the study expands to 8610 games and some 492,849 shots on goal, a pretty significant sample size!

Outshooting: 8278 GP, 4127-2985-602-564, 9420 points, .569
Outshot: 8278 GP, 3549-3384-602-743, 8443 points, .510
Shots Even: 664 GP, 306-236-52-70, 734 points, .552
Total: 17280 GP, 7982-6605-1256-1377, 18597 points, .540

Interesting to note that MacT’s career percentage of .540 is exactly the league norm. Of course stupid medians like .540 are courtesy the fluctuating Bettman Point rule, which have corrupted the meaning of points percentages. Before the lockout when Bettman points were awarded just in decided overtime games, there was an overtime goal in about 1 game in 20, for a league mean Pts% creeping beyond .525 as coaches caught on. In those four years 2000-04 the outshooting team had an edge in points percentage of .554 to .500. Since the introduction of the shootout ensuring Bettman points in all regulation ties, the median has soared to about .555, and again the outshooting teams have a small edge, .589 to .524 from 2005-08. Overall the edge goes to the outshooting teams, .569 to .510. That’s significant but hardly overwhelming, +/- .~030 on the normalized points percentage. Enough to make a difference? Sure. Enough to make The difference? Meh … I think we could find other factors which had a much stronger correlation with winning.

Conclusion #1: Across the NHL there is a mildly positive correlation between outshooting the opposition and outscoring the opposition.

Conclusion #2: The Edmonton Oilers are an exception to this rule, consistently performing as well or better when outshot than outshooting.

Both of which seem fairly consistent with my speculation at a much earlier stage of this research during the Horcoff thread, when I said about one of my Oiler studies:
I am frankly shocked by the strength of the inverse relationship of all of those shot attempts and actual goals. I would expect — and would still expect to find on a league-wide basis — … a modest positive relationship here.

And so it is. By my “odds indicator” it’s a 53:47 split, indeed positive but definitely modest. Which suggests to me that we should only take modest confidence in Corsi numbers as a measure of individual players; it measures shots for and against, but shots only mildly drive results. I’m very handy with numbers but I’m not a trained mathematician; maybe one of you whizzes on the cutting edge of modern hockey theory could design a linear regression analysis and prove something else, I dunno.

By all means tell me the flaws in the method – I can give you a few already: it identifies shots but not Corsi numbers; it doesn’t differentiate for special teams shots; it doesn’t differentiate by “how many” shots – but I think all of those objections are diminished by the huge sample size. The team that outshoots on the night will generally outshoot at evens, and they will generally out-Corsi. Not always, but the “corrections” will go both ways w.r.t. results, and the overriding percentages should hold pretty well for your metric of choice is my guess. But I will NOT be doing such a second-order study unless somebody has already collected the raw data.

I’m not digging into the nitty gritty about game states here, which I think is incredibly important.

Absolutely right, MC. Without this information what remains about shots on goal is flawed IMO. I personally expect the game state w.r.t winning/trailing team is probably the most important consideration here; in fact it wouldn’t surprise me if the team trailing on the scoreboard generates a clear majority of shots. The inferred 6% winning advantage enjoyed by the outshooting teams might largely occur in that portion of the game that they are acquiring the lead. But I don’t have the computing chops to generate that type of info, and I don’t blame you for not wanting to either. Perhaps one of the computer-savvy types who are already mining the play-by-play logs for Corsi numbers and shot quality and such can split shots data into game states, a simple sort into “ahead”, “behind” and “tied” would be most instructive. How about gross shots league-wide when the margin is exactly one goal? Two goals? If it’s one goal in the last ten minutes? Etc. (So many ideas, so little talent.)

A secondary game state consideration would be special teams, as both MC and Slipper mentioned last time. A powerplay Shot has a little better chance of going in. But usually the outshooting team will also have the most ES shots, so I’d wager this is a second-order effect. Maybe the team with more powerplay shots wins 55% of games instead of 53%, but I’d be surprised if the difference was even that much. I’m certainly prepared to be contradicted, I’m not married to any particular theory. But my best guess is that in the context of any given game, shots are shots, and the team with more of them regardless of manpower situations has a small but hardly overwhelming advantage. (Unless they’re Detroit, an exceptional team in every respect.)

Something that I found very interesting: the changes introduced by the NHL in 1998-99 in terms of adding loser points have largely benefitted the teams that get outshot.

Excellent induction, Tyler. I noticed something along somewhatsimilar lines in my review since 2000. Of course the Bettman Point was in existence all those years. What is startling is how the outshooting team fares much better in overtime than it does in regulation. I have split it into two segments, without and with the shootout which of course muddies the picture further still, since it has not a hell of a lot to do with the game itself.

From 2000-04, the outshooting team won just 1919 of 3622 games decided in regulation (.530), but 296 of 509 games that were decided in OT (.582). (Of course, slightly over half of OT games were not decided at all) Since the lockout that effect has scarcely diminished, with the outshooting team winning 1465 of 2647 games decided in regulation (.533), but a significantly better 447 of 798 games decided in OT or the skills contest (.560).

What happens in OT? Does the team that’s been outshot throughout the game get tired? Are they happy that they achieved their one point and are more susceptible to a beer-time mistake? Does the superior team turn it up a notch themselves, finally going hard for their second point with one securely in the bank? Are outshooting teams generally good 4-on-4 teams? Good shootout teams? Or what? There are lots of possible explanations, or more likely, some unholy creole involving several ingredients, but once again there is certainly evidence that the Bettman Point has corrupted the way the game is played.

In the seasons before the loser point was introduced, outshooting teams averaged 1.108 pts/g; in the seasons since they’ve averaged 1.139 pts/g. For teams that get outshot, they averaged 0.892 pts/g prior to the introduction of the loser point and 1.011 points since.

So before the Bettman Point the outshooting team took 55.4% of the available standings points, and exactly 53.0% (there’s that number again) of the awarded standings points since. That margin is closing if anything; in 2007-08 the outshooting team took just 52.0% of the awarded points, down from 52.7% last season. Yet more evidence that while the league talks the talk about promoting offence, the “system” they have patched together in a particularly grisly vivisection of competitive craziness and corporate counterintelligence, doesn’t actually reward it. And coaches ain’t stupid, esp. our own Mr. MacT.

the rule changes made by the NHL with respect to OTL have been designed to make the league look more competitive than it actually is

I couldn’t agree more, MC. Add another couple of logs to my platform that the Bettman Point has compromised the competitive integrity of regular season hockey.

Bruce: Perhaps one of the computer-savvy types who are already mining the play-by-play logs for Corsi numbers and shot quality and such can split shots data into game states, a simple sort into “ahead”, “behind” and “tied” would be most instructive.

There is at least one. Javageek’s shot quality numbers (SQF/SQA) take into account the distance to the end boards, game situation (EV/PP/PK), shot type and the score (leading/trailing). There are separate tables for EV/PP/PK. The basics behind it all are described in Alan Ryder’s shot quality doc.

I’m not sure if it’s wise to include a leading/trailing factor. While it’s true that leading might lead to higher quality chances as your opponent takes more chances to tie the game, I think the quality for/against factor will be skewed by good/bad teams. Good teams often lead and bad teams often trail. More of the good teams’ inherent SQ will be built into the leading factor and more of the bad teams’ inherent inabilities will be built in to the trailing factor. I think this is responsible for screwy results sometimes, like game 5 of Det vs Dal. The Wings badly outshot the Stars but the expected goals favour Dal because they led for most of the game.

If you want to include a leading/trailing factor in the SQ calculation, I think it has to be normalized to isolate the effect of an average team leading/trailing an average team. I don’t think Javageek’s calc does this.

Using the distance to the end boards is a huge problem. A shot 12′ from the boards can occur in the crease 1′ from the goal line or can come from the corner 40′ from the net. Correcting the distance factor to take the angle into account should increase the difference in SQF between teams who produce tap-ins from teams who shoot from anywhere.

There are problems with the raw data too. Something weird is going on with the Sharks, and Ryder’s paper says the Rangers’ stats collectors are crosseyed. I’m sure you’d find different ’standards’ for different shot types in different rinks.

If you take it at face value, Javageek’s SQF is just as good as shots for as a predictor of goals. It suggests SQF and SF are roughly equal contributors to goals league wide. It doesn’t say the same about goals against - the better part of good defense is preventing shots against, not reducing shot quality.

Some measuring sticks: It’s clear Detroit dominates at EV with shot differential, while Pittsburgh does it with quality chances.

Everyone’s probably already seen this - Desjardins’ effort at the same thing, only it takes the angle into account.

Here is a sample of the raw data from ESPN. The file has the x,y coordinates for each shot, length, type and game situation. This has potential.

Bruce on flaws: …it identifies shots but not Corsi numbers…

I thought the value in the Corsi number was in the short view - a single game or small sample of games. The problem with GF/GA is sample size, even over the whole season. In the short term you might have the same problem with SF/SA, so you extend the view to include the misses and blocks. Over the long term, i.e. a full season, simple SF and SA should be as good or better a determinant of GF/GA than the Corsi number.

Javageek did a post on Corsi numbers, saying pretty much the same thing.

“5 of the 7 years the Oilers had a better record when being outshot”

So, does that tell me that MacT’s teams play a style that allows low percentage shots and stresses rebound control?
And counterpunching?

Tyler and Brue, I’ve just now — on the morn of game one of the Cup finals — read all both of you wrote on this issue.

Excellent hard work by both of you. Simply outstanding, and the kind of thing that was needed to get to the bottom of the great and rancorous debate over the relationship between shots on goal and Corsi numbers and the like. Thanks for putting in the time and effort here.

Here is what Bruins coach Mike Milbury said about the 1990 Oilers, a team that got outshot on its way to Cup success.

“They had the team speed but more importantly, the ability to handle the puck at high speed and to finish off their plays at the blueline. People talk about our inability to get second shots but Edmonton didn’t need second shots, they were scoring on their initial rushes”

The thing that really bothers me about it is what this means for SV%- I’ve always thought that it’s just about the best metric we have for goaltenders, but if the edge for the outshooting team is so small, how valuable is it really?

Jonathan: A very good question. I’m always wary of any shot-related metric carrying too much weight in analysing individual performance, be it Corsi number or Sv%, on the simple premise that not all shots are created equal.

Certainly Sv% (and shot quality neutral Sv%) is one of the better metrics for goalies. However, circumstances differ from one team/coach/system to the next, and I am convinced the goalie himself makes a contribution to the number of shots he faces through puck retrieval and distribution, rebound control and the like.

As it happens I’ve been in the middle of a huge debate on this issue with the Contrarian Goaltender on his provocatively titled blog “Brodeur Is A Fraud”. If you’re interested, check out the recent post “Top All-Time Goalies” and subsequent comments.