mc79hockey.com

Yeah, i don’t know, I’m pretty skeptical - how much are those numbers riddled by small sample size and selection bias errors. Also, where’s the correlation of these extra points to winning? No adjustment for goals/shots against, and no adjustment for quality of minutes?

tango:

There seems to be a number of old Oilers in that list. Gretzky, Messier, Kurri are mentioned, maybe there are more in a bigger list?

Is that anything but fluke, or do you think there was something about that old Oilers team that allowed them to retain more of their regular season offence?

Is that a guess, MC, or do you know that to be true?

I guess I wonder if those teams consistently had way worse goaltending/defence than other divisions 3rd and 4th place teams, so much so that they skewed the Oilers results? How division heavy were the schedules back then?

Tango, I’m not sure what you’re getting at. I mentioned concerns about small sample size and selection bias, and you brought up career leaders in save percentage relative to league average?

Just so we get straight to the point, why don’t you just tell me what your hypothesis is wrt your article. I totally understand you had space concerns, and you probably also wrote to cater to a fairly wide audience, so, to be clear, I don’t doubt that you may have valuable insight.

Let’s concoct the following hypothetical. Say we take Regular Season Mark Messier and compare him to Noah Heart, a fictitious player of EXACTLY equal value as Mess. (I am defining “value” solely as a player’s ability to influence his team’s goal ratio - i.e GF/GA - and I am pretending that we have a precise way of measuring this. Hey, it’s my hypothetical!)

Now assume both players’ teams make it to the playoffs. Here is my question: If you were a wagering man, would you wager a large sum of money that Mark Messier would outperform (again, according to the aforementioned “value” rating) Noah Heart in the playoffs? I’m honestly curious about your opinion here.

how do we know that these players are not there simply because someone has to be, just by the rules of the universe? Even guys roll the dice, there will be crazy looking stretches of 7s and 11s from some guys and equally madass looking runs of 2s, 3s and 12s from others.

Sunny, that should be “Enough guys roll the dice … ”

Jebus, I really should reread before posting.

Tangotiger:

I am back online tonight and was going to post a friendly “I see there have been some other posts as well - do you still need that article summarized?” or something to that effect. I figured the link wasn’t working for you. Rereading Tyler’s original post and making the obvious connection between the author’s last name and the name you post by, I am left to believe that you either a) are extremely skeptical of people’s intelligence over the internet, b) don’t respond especially well when people criticize you without supporting arguments, or c) an asshole. I’ll take you at your word to divide up the proportions yourself.

As for the article, what I don’t like has already been touched on by Vic: Without consideration for the general population, how do we know that these players are not there simply because someone has to be, just by the rules of the universe? Even guys roll the dice, there will be crazy looking stretches of 7s and 11s from some guys and equally madass looking runs of 2s, 3s and 12s from others.

This was my immediate reaction to the article. You look at a normal distribution of players and you make it big enough, you’re going to find Messiers and Pisanis, you’re going to find people in the middle, and you’re going to find people who drastically underperform their seasons’ pts/g during the playoffs.

If you can prove that the variation isn’t just noise and statistical randomness, I would love to read it. Having a guy like Messier on your list passes the sniff test but IMO your methodology includes too much noise (again, Vic’s word).

Tyler: Do you think I crossed the line? I sincerely do mean the last sentence I typed in my criteria paragraph - I am by no means calling anyone names but I do find asking a guy to summarize an article he clearly just read a bit brash. I also wrote b) in part just to point out that my original post was support free.

Lastly, I will happily credit anyone I haven’t previously read with a solid writing history when recommended by someone else whose writing I respect - you in this case or Lowetide in the case when I thought a John Short article was fluffier than it was significant. All I ask is something similar in return and I consider my posting history to show that I am thoughtful, respectful, and generally polite to those I disagree with. Perhaps my replies to this topic don’t reflect that way.

“Is that anything but fluke, or do you think there was something about that old Oilers team that allowed them to retain more of their regular season offence? ”

The analysis was limited to the method, which was to look at career totals. So, there is a bias if a disproportionate number of playoff games are played while a player is at his peak (his 20s). And this would affect the Oilers likely the most. Similarly, Yzerman doesn’t end up looking good in this, since a disproportionate number of his playoff games were of him in his 30s, while his great regular season scoring was more in his 20s.

***

“I suspect that you are looking at noise. ”

All samples have noise. Even when Gretzky scores 92 games, some of that is noise. Most of that is his talent, but some of that was simply him catching more lucky breaks than unlucky breaks.

***

“If you still weren’t convinced, you could build a model and plot the distribution expected by random chance alone atop that of the actual players. I suspect that they’re going to be right on top of each other, because there is nothing there. ”

Clutch does exist in baseball for the simple reason that humans respond differently to perceived stress. This is even more real in hockey. The question however is always how much does the purported metric measure skill as opposed to noise.

This is true even with regular season scoring.

***

“If you were a wagering man, would you wager a large sum of money that Mark Messier would outperform (again, according to the aforementioned “value” rating) Noah Heart in the playoffs? I’m honestly curious about your opinion here. ”

You would be crazy to wager a large sum, even if you are convinced that Messier has a higher talent level in the playoffs than Noah Heart. This is because in any sample, random variation plays a role. And the smaller the sample, the more random variation’s impact can be felt.

Suppose that you have a die. If I roll a 1 or 2, I win. If you roll a 3,4,5,6, you win. Clearly, the odds are stacked in your favor. We play a “best 2 out of 3″. Three rolls, winner take all. How much would you bet? Would you bet your house on three rolls?

Now, suppose instead of a best 2 out of 3, it’s a best 1000 out of 1999. Now what do you do? Well, now you bet for sure. It’s going tobe an easy win for you.

Somewhere in between, you have your doubts, and so, the amount of your wager will depend on the chance that you will. Even though you KNOW for certain that the odds are for every roll.

For player performance, we don’t even know that. We can estimate the talent level of Pisani or Kurri. Around that talent level is a certain amount of uncertainty. That uncertainty is directly linked to how much of a sample you got to observe until that point.

But, as I said, even once you know, or can estimate, the true talent level, you still are at the mercy of the number of games to play in the future. Ovechkin can take 15 shots in a game, and not score a goal. That doesn’t mean much at all.

I see a bunch of typos I made, like saying “games” instead of “goals”, among others. If you read the post a bit fast, you won’t notice anything is amiss.

“Suppose that you have a die. If I roll a 1 or 2, I win. If you roll a 3,4,5,6, you win….We play a best 2 out of 3…..Now, suppose instead of a best 2 out of 3, it’s a best 1000 out of 1999. Now what do you do?”

My expected value (or “EV” as we call it in the gambling world) is identical in both scenarios. You’re offering me even money on a prop in which I am a 2-to-1 favorite, so I expect to make 33 cents on every dollar wagered. It doesn’t matter how many trials we run.

Now, obviously a smart gambler makes ANY bet keeping in mind risk of ruin. But when I asked if you would wager a large sum of money, I meant large as in “number of dollars” (as a cheerful metaphor for your amount of confidence in the bet), not large as in “as a percentage of your total bankroll” (in some sort of defiance of the Kelly Criterion or something).

Forget the dollars. Just ballpark how confident you would be in predicting that Mark Messier would outperform Noah Heart in the playoffs?

(PS - Spam: If any of you guys are poker fans, my new book comes out next week. Click my name for more info.)

I think we are understanding each other. Perhaps I should have used a different word than “confident” because I didn’t literally mean “statistical confidence intervals”, which is essentially what you’re talking about. Variance. Yes, it exists. But expectation does NOT change. When I originally asked if you’d bet Mess vs. NoHeart in the playoffs, I was essentially asking you to handicap it.

If you think Mess’ 50-50 no-edge goes to a 52-48 edge, or 60-40, or whatever, that’s what I’m asking, and that number does not change no matter how many games they play. Yes as a simple binomial outcome of “ahead” or “behind”, the favorite is more likely to be “ahead” as the number of trials go up. But I thought that was understood. Bet and handicap are synonymous to me. If it’s a good bet, you make it. Yes the AMOUNT you bet, as a percentage of your total bankroll, is dictated by the host of Stats 101 formulae we’ve already alluded to. But I wasn’t implying that you’d be burned at the stake if NoHeart outperformed Mess after one playoff game, I was just trying to get an idea of how much you really think Mark Messier was capable of playing better in “clutch” situations than a player of exactly equal skill level, and if you felt the data you’ve looked at accurately supports that.

From your last few posts, it sounds like you do really believe in clutch. I don’t NOT believe in it. I recognize that absence of evidence is not evidence of absence. I just think no one has really sufficiently shown it to exist, and imo neither did you with the Mark Messier article.

Btw, I’ve been thinking recently that putting in golf would be a decent starting point for the clutch debate. I’m talking about putts by distance and situation (i.e. - more “meaningful” points in the tournament, for the win, etc) on the PGA tour. It seems like we could get a decent sample, and it’s the type of thing that would have minimal outside influence (i.e. - no teammates, etc) and also a pretty clear-cut outcome range (”putt made” or “putt missed”).

Sunny

Yeah, snooker would be another high stakes game where I imagine it comes into play. Having said that, the leisurely pace of baseball has the prerequisite “time to think”.

My best guess at cracking the nut: Taking a big timespan with baseball, like eight years, then using just players who played a significant number of games over this period … then isolate the clutch situations by whatever metric you feel most confident in. Then grab 100 clutch situations randomly for each player repeatedly, so you have thousands of data points building your observed distribution.

Then build an ability distribution iteratively using the beta form, until you’ve got something that works (i.e. you take your brand new ability curve and parlay it through the universe’s luck curve and plot the result … if it isn’t the same as the observed distribution you built in paragraph one, try again. Ad infinitum, or until you deem it ‘close enough’

At the very leasy that will give us the shape of the animal we’re trying to kill. And the process may lead to different answers, or at least lead to better questions to ask.

Even J. Albert walked away after associating (and this is by my notoriously dodgy memory) four years of runners-in-scoring-position divided by empty-bases-nobody-out stats associated with the same for the following year. I think the real effect was only in the .15 range using pearson correlation. So he walked away from it. A shame. Material or no, it’s an interesting topic.

By the way, I ran a clutch project here:
http://www.hardballtimes.com/main/article/with-the-game-on-the-line-i-want/

And the results are here:
http://www.hardballtimes.com/main/article/the-color-of-clutch/

do you think there was something about that old Oilers team that allowed them to retain more of their regular season offence?

It’s a sample bias. Except for Glenn Anderson, those guys played an disproportionately high percentage of their career playoff games with the Oilers before the group splintered and they all wound up on lesser teams (or, this time of year, the golf course).

I wrote a comment about this on Mirtle’s post “Finding the playoff underperformers” a while back which makes the case statistically about how pronounced this effect can be. So happens to mention three of the great Oilers referenced in this comments thread.

This is especially so if a player’s career spanned eras of significantly different season-over-season scoring levels. After his last playoff game, Mark Messier played 7 seasons and 484 regular season games as an old man in the Dead Puck Era. The result was an already-great ratio ballooning to out-of-this-world as his per-game rates plummeted in the regular season while remaining constant in the playoffs.

I did say this:

The analysis was limited to the method, which was to look at career totals. So, there is a bias if a disproportionate number of playoff games are played while a player is at his peak (his 20s). And this would affect the Oilers likely the most. Similarly, Yzerman doesn’t end up looking good in this, since a disproportionate number of his playoff games were of him in his 30s, while his great regular season scoring was more in his 20s.

In my view, it is impossible to effectively introduce an adjustment for the scoring levels of the era, as well as the number of games played in a player’s aging curve, and do so in a 250-word article.

All I’d end up doing if I did this would be to present some sort of black box that will be a “trust me” kind of numbers.

As it is, my work is reproducible.

And more importantly, it is the first step. There is no one anywhere more conscious of adjustments and being aware of context than me. So, I don’t see the point to continually bring this up, since I’ve probably brought this point up in hundreds of my post to tens of thousands of readers.

TT: Fair enough. I had noted your Yzerman reference in the comments section and had intended to mention it was on the right track.

I do think it’s a hugely important point. I don’t know how I would handle it in a 250-word article to tens of thousands of readers though. I know in my work as a science popularizer one has to be very careful in where one draws the line between “too much information” and “not accurate enough”, so I readily empathize with the dilemma you face.

No disrespect intended.

I keep coming back to this because I have a sinking feeling that this is the kind of thing the Oilers just might buy into. It’s not easy being an Oiler fan.

Obviously there is a lot more to hockey than just points, and hockey is a luck soaked game.

Still, sunny’s terrific question cuts to the core of the matter. Essentially “would Mike Smith bet his own money on this?” Since he is presumably asking GMs to bet their owner’s money (and the mental health of the fans) on this stuff, it’s a wholly reasonable point.

Since what we’re talking about here is more directly related to winning hockey playoff pools than hockey games, it should be framed that way. Then again, if mudcrutch79 had identified the subject as “talking about tips to win your playoff pool” as opposed to “talking about playoff clutchness” he would have drawn less commentary.

If I’ve understood this properly, then Mike Smith (Coleman Analytics) believes that (past playoff scoring)/(past regular season scoring) is a strong indicator of future playoff scoring. I’m sure he’s wrong.

Now common sense things like injury status, who is playing on Crosby’s line, etc. will always be important. But as a fair test, if we’re predicting playoff points per game for a large group of players in any one season, and we use regular season points per game from the past three seasons as the starting point …

If Guesser A corrects this number using previous playoff scoring rates only.

If Guesser B corrects this number using points since the all star break only.

If Guesser C corrects this number using expected points that season (what a player’s points would have been had his on-ice shooting% been at his average from the past three years).

Would that be a fair test of the methodology? It would seem to be to me, but I’m wide open to changing it. I may well be misunderstanding something here as well.

And if additional information were not allowed by any of the guessers A, B or C (no injury or line change info, no matter how obvious and important … so that the playing field is level for all)

So who would you want the rent riding on? A, B or C?

But Tom, my point wasn’t that you HAD to bet on one or the other, the point is WOULD YOU? I.e. - Are we talking about edges that are even worth talking about? I.e. - Big enough that you’d have an edge betting on?

If you sort of, kind of, think you maybe might just handicap Mess’ edge over NoHeart to go from 50-50 to 51-49, how is that worth a GM’s time or money to pay any attention to?

On the other hand, you take some of the research that guys like Vic, Tyler, JLikens, myself, and others have been doing on possession, faceoffs, territorial indicators, inflated percentages, etc, and I bet you’re gonna find pretty big inconsistencies in the market wrt player value.

Admittedly I don’t know for sure what sort of level these GM’s are operating on in terms of hipness to stats. But I do know that

a) I seem to regularly observe NHL teams make what I consider bad deals.

b) A good friend of mine is the doctor of a current NHL GM and has discussed the issue of statistical analysis with him (at my request lol), and the GM wasn’t too interested in the topic.

c) I bet on a lot of hockey games, and while betting markets are still the most efficient and predictive imo, I think there is a good amount of area to exploit.

Hit submit too early. The point of that was just to say that it’s no surprise that there’s inefficiency in the NHL player acquisition market as far as building a winning team. Especially when a low-revenue southern team might have different financial incentives than a Western Canadian one…

Unfortunately, teams really do want you to work for nothing, or at the very least, for less than the law allows. That’s not a recipe for success.

I wonder how many NHL teams know which opposing players take defensive zone faceoffs? Or which opponents are on the ice at the end of close games? It seems like that’s what Smith is selling…

Right.

You can’t take the analysis so far that you put the team in a position of disagreeing with your analysis, which has a cascading effect of rejecting the underlying data.

You present data, you make some adjustments, they ask questions, you make more adjustments, they bring up more parameters, you make more adjustments. You present your final report, and then they give it a “tie-breaker” weight relative to how they originally were thinking.

For example, say you have Dave Andrechuk, and a similar (but younger) player, and they have the same stats. They had considered releasing Andrechuk, but not the younger guy. You present data that shows that both players are not performing well. That’s enough for the team to release Andrechuk, but does little for how they perceive the younger guy.

It’s close to confirmation bias, but not totally. Sometimes the analysis highlights some surprises, which simply means the team will investigate that player more, to see if the underlying physical tools actually matches to his statistical performance. Dan Boyle might be a good example of someone that shone pretty bright in our analysis, but wasn’t considered (at the time) at the top level.