I recently read a great book by a guy named Dan Gardner, a writer with the Ottawa Citizen, called Risk: Why We Fear Things We Shouldn’t - and Put Ourselves in Greater Danger. I think of him as being sort of a less-pop, more substantive Malcolm Gladwell type - I’m not a huge Gladwell fan, but I do support the idea of critically examining ideas.
He’s got an interesting background, having worked for a few years for Mike Harris’ government in Ontario. If you take some time and read through the archives of his blog or his seemingly defunct website, which I recommend doing, you kind of a get a sense that he’s a guy who has, if not a libertarian bent, at the very least a strong bias towrads evidence, and skeptical of government policy in a lot of areas, particularly with respect to drugs, prostitution and law and order issues.
It’s a bit surprising to me that he lasted as long as he did in the Harris government, given that it was sort of at the forefront of the Americanization of the Canadian conservative movement and strikes me as being closely tied to our current federal government, which seems to govern on the basis that the only thing more important than actually solving a problem is being seen to have solved a problem by a sufficiently broad segment of the electorate that you can ensure your re-election. It’s a variant of the Oilers’ modus operandi.
There’s a lot of great stuff in Gardner’s book and I’ll probably touch on some of it from time to time in the future. His thesis is basically that people are lousy at correctly perceiving and responding to risk, something that I agreed with before and agree with more strongly now that I’ve read his book. The point I’m interested in right now though is a point that he made about science. Gardner wrote:
Unfortunately, the language of science is the opposite of the simple, definitive statements the media want. In science, all knowledge is tentative, every fact open to challenge. Science never delivers absolute certainty. Insteda, facts are said to be known with degrees of confidence…Uncertainty is so central to the nature of science that it provides a handy way of distinguishing between a scientist tlking as a scientist and a scientist who is using the prestige of his white lab coat to support political activism: Look at the language. If a scientist delivers teh simple, unconditional, absolutely certain statements that politicians and journalists want, he is talking as an activist, not a scientist.
Now, I’m not going to pretend that what I do here is anything approaching science, but I’m probably guilty of this from time to time, although perhaps not as often as I’m accused of, complete with supporting misrepresentations of what I said. I have a suspicion that if Nikolai Khabibulin plays 60 games of .915 hockey, I’ll hear that I said that was impossible, which is not at all what I’m saying. With that said, one of the things I’m struck by with the Khabibulin signing is the complete absence of any evidence that says, “Yeah, this is a good idea” that doesn’t involve looking at things that are heavily team influenced, like Stanley Cups and playoff success.
In light of some of the comments to my previous post, as well as the inevitable signing of a guy with a lot of success to a complete steal of a contract, I thought it’d be interesting to explore the contrast between Martin Biron and Nikolai Khabibulin a bit more. In a previous post, I noted that they’ve both faced a lot of shots since the lockout and that there was a substantial gap in their performance, with Biron having faced 5605 shots with a .912 save percentage, while Khabibulin faced 5628 shots and stopped them at a .904 clip. A 47 goal difference or so over five years, which is a lot - you’re basically talking about two wins a year.
Save percentage is fickle enough that I wondered whether that’s a difference that actually tells us anything about a goalie. I’m not the type of guy who gets all that worked up about save percentage differences over the course of a single season, just because of the randomness involved. If we were talking about a single season with a difference like that, I wouldn’t really care - I’d be interested in the long term.
I set up a study to look at this. It involved going through each year for which there’s at least four years of old save percentage data and identifying pairs of players where a) both goalies have faced at least 5,500 shots over the preceding four years, b) there’s a gap of between .005 and .010 between them and c) they both faced at least 2,000 shots over the next four years. For those interested, the scope of the study covers players between 1983-84 and 2003-04 - obviously, I don’t have the four years I need at the front end to go beyond that.
I was able to identify 247 pairs of goalies who met the criteria. One of my favourite points here is that Dominik Hasek only made two pairs - his save percentage was so much better than the second best guy throughout his time in the NHL that he simply didn’t have any comparators for much of the study. When THN publishes their next “Best Hockey Players of All-Time” magazine, he won’t rank inside the top twenty and that’s criminal. He should, in my opinion, be part of the discussion with Gretzky, Orr, Lemieux and Howe.
Anyway, after I identified my pairs that met the criteria, I proceeded to score the thing. Basically, if the “after” save percentages were within +/- .004 of each other, I scored it a tie. If the goalie with the better save percentage coming in posted a save percentage of .005 or more above that of the fellow with a lesser save percentage, I called it a win. Otherwise, it was a loss. The players identified as having the better save percentage posted a record of 104 wins, 65 losses and 78 ties.
In the circumstances specifically faced by the Oilers, they were considering a player who is five years older than another player. Given that the 5,500 shot pre-requisite is going to knock a lot of players out of the running, I’m not going to be dealing with a lot of particularly young guys. So I took a look at only those pairs in which the player with the lesser numbers was at least five years older than the other player. Again, pretty decisive outcome in favour of the player with the better numbers 15 wins, 7 losses and 17 ties. Sean Burke is actually responsible for four of the losses - he had an atrocious .876 save percentage in 1992-93 and was then below .906 just once in the next eleven seasons.
Looking at this, I don’t see a lot that suggests to me that Khabibulin is the better bet in terms of save percentage for the next four years. It’s another slice of data, another different way of looking at the problem but again, it doesn’t suggest that the Oilers made a smart bet here. The fact that they felt the need to put so much on what looks like a poor bet is all the more perplexing, particularly when the guy with better odds was available for less.
Khabibulin may well be in the minority who ends up beating the odds. I don’t see any soft stuff (injury record, save percentage trends, extraneous circumstances, etc.) that suggests that he will be but that’s not the point I’ve been driving at since they signed him: no matter how you look at it, this just looks like a terrible bet. The Oilers (the hockey ops people anyway) just don’t seem to be very good at identifying and managing risk. Cheering for them is not unlike having an emotional investment in a guy at a blackjack table who hits on 19 with the dealer showing 6. You’re left hoping that he pulls a 2 which, even if it works out, is kind of depressing to cheer for.
I swear that when I posted this at HF (reply #39), I had yet to read this post.
Creepy.
It strikes me as especially weird since I included a link to one of your posts in that reply.
that link doesn’t seem to have worked.
http://hfboards.com/showthread.php?t=659400
What is Niklas Backstrom’s value, given his high SVPCT? A legitimate $6M goalie? Or a product of his defense?
We have data that seems to suggest Biron played behind very good defense and Khabibulin behind very bad.
Have you written about this Gabe? Do you have a link to something current? I’ve always been a bit uncomfortable with the quality of competition of stuff. I also think that it tends to wash out in the longer term.
which seems to govern on the basis that the only thing more important than actually solving a problem is being seen to have solved a problem by a sufficiently broad segment of the electorate that you can ensure your re-election. It’s a variant of the Oilers’ modus operandi.
Maybe it’s cuz it’s 4am. And maybe it’s cuz I’ve had a few drinks - but this sentence struck me as particularly effective and salient.
Awesome.
Tyler, what was the average number of shots faced by the goalies in the “pre” and “post” periods of your study?
“If a scientist delivers the simple, unconditional, absolutely certain statements that politicians and journalists want, he is talking as an activist, not a scientist.” -Gardner
- OK, lets apply this to your own post.
“I don’t see a lot that suggests to me that Khabibulin is the better bet in terms of save percentage for the next four years.”
- Most scientist would support a linear time model, that allows for some predictions but is not deterministic. I don’t see the accompanying data that points to Bulin’s predictions of failure/risk: if it is in another post please point that out to me. I believe that you are setting up a deterministic model of save percent the reaches into the future by extrapolating from the past: kind of like weather-man science, aka not the most effective model for reducing risk (think about planning BBQs or Beach Days). If you where using a abstracted mathematical model based approach, like astrophysics, I would be in favour of your methodology and value this post more.
“Khabibulin may well be in the minority who ends up beating the odds.”
-Again, I don’t see the odds of him being a failure or a success. This statement is the most ‘activisty’ rather the scientific. What is a soft stuff, that allows him to beat them? IF it is things like injuries, could that not be calculated into Odds? Is not ‘extraneous circumstance’ another way of saying team play: something you dismissed as effective to base goalie’s value on in the first paragraph.
Overall, I normally agree with most of the things you write (or at least have a grudging respect for those I disagree with), but in the Bulin Wall case- this summer- you are being to sound more like a Creationist Scientist or Al Gore clone. I am not convinced, that we can judge a goalie’s (or any players) value four into the future using comparative models of evaluation. That the model you are suggesting is inductive and not deductive, hence it is not scientific but rather it is historical.
Please keep up the great work, as I do enjoy reading it.
Thanks for bothering to attempt to model the Oilers goaltending situation Tyler.
Inductive or deductive, what chaps my ass is that late in the AM on July 1, when Khabby and Biron were both still available and Anderson and Roloson had been signed. The Oilers brass did not recognise that they were the last team offering NHL starting minutes. Even if Khabby went back to Russia with a KHL team, surely a week later we could have made a decent (3years, $9MM) deal with Biron on that fact alone. (The risk that you are taking is that Biron chooses anything else over Edmonton at that point - a supreme kick in the teeth.)
I now find myself in the position of cheering for a team run by men who have allowed Charles Wang and Garth Snow to look astute at their expense.
Tyler, what was the average number of shots faced by the goalies in the “pre” and “post” periods of your study?
6377 and 5084.
@B.C.B.: Look at it this way: I started with a theory, which is that a difference of save percentage in the 5 to 10 point range over 5500+ shots tells us something about a guy going forward that it wouldn’t over the course of a single season. The results, to a certain extent, seem to bear that out. It’s not an absolute thing but there’s seemingly something there.
As far as the soft stuff, I’m thinking mostly about stuff like demonstrated record of health. I haven’t tested that but it seems more likely to me that the guys who are going to be able to fight off the ravages of age are guys who don’t have nagging injuries.
I am not convinced, that we can judge a goalie’s (or any players) value four into the future using comparative models of evaluation. That the model you are suggesting is inductive and not deductive, hence it is not scientific but rather it is historical.
How do you suggest that it be done? I’m as aware of problems with the methodology as anyone is but it seems to me that it is, at the very least, valuable to know this sort of stuff. What do you think that the Oilers are doing to project that he’ll be useful four years from now? I strongly suspect that there’s a group of people there who think that goalies are playing better when they’re older now than they used to, because of their experience with Roloson, and that Khabibulin must be better than Roli because he’s won a Cup.
Tom has a post and discussion thread up about this.
@Kent: I think it’s a problem that all governments have to a certain extent. I feel like it’s particularly acute in the case of the Tories, although I’m not sure why I feel that way - it may be because I’m a nominal Tory who is now thinking awfully hard about switching allegiances. I really don’t like the American conservative movement and the Tories seem to me to be on that course.
Most of the Liberal bad policy just strikes me as sort of inoffensive mumbo jumbo (Green shift aside). The Tories seem to have this bad, regressive law and order bent that they exacerbate by a love of screwing with the tax system to curry favour with defined segments of the electorate regardless of whether its sound policy. I recognize that “sound policy” is a bit of a subjective idea in a lot of ways but they go far enough on enough issues that I think a lot of reasonable people who might otherwise be inclined to support them can agree on the point.
Looking at this, I don’t see a lot that suggests to me that Khabibulin is the better bet in terms of save percentage for the next four years. It’s another slice of data, another different way of looking at the problem but again, it doesn’t suggest that the Oilers made a smart bet here. The fact that they felt the need to put so much on what looks like a poor bet is all the more perplexing, particularly when the guy with better odds was available for less.
I don’t understand how you can once again make this claim when it’s simply glossing over the way free agency works. Biron was reportedly seeking $5 million a year when free agency opened, and the fact that he signed for $1.4 million with the Islanders does not mean he would have done so with the Oilers, and certainly doesn’t mean he would have done so on July 1. In my estimation, Biron signed for so cheap and to a bad team to get himself traded to a contender at the deadline, and to almost ensure that he would be getting into playoff games. If he manages that, he’s going to get a solid contract come next July, probably for around the amount Khabibulin signed for. When Khabibulin signed, most teams looking for a goalie still hadn’t found one; Biron’s price might’ve still been higher than Khabibulin’s.
Why did the Oilers have to sign someone on July 1? Why could not they play these guys against one another? Negotiation is all about leverage.
Why did the Oilers have to sign someone on July 1? Why could not they play these guys against one another? Negotiation is all about leverage.
they didn’t. but as a team on the cusp of the playoffs, they certainly had much more pressure to sign someone than someone like the islanders. however, based on the price they paid, they almost certainly said khabibulin is their guy and sought to get him by whatever means necessary.
again, i am not exonerating the oilers, the khabibulin signing is at least bad, but comparing it to the biron deal isn’t exactly fair. to wait for bargains in free agency typically leaves a team on the outside - this is the first year where so many good players are still unsigned nearly a month in.
Triumph
The comment from the Oil that Khabby was their guy is worthless. The alternative statement is ‘We really wanted ______, but he would not come here, so we snapped up Khabby.”
The question is, if Biron signed for $1.4 million to back up Roli on July 22, what would Biron have signed for to start for the Oil on July 15?
Would that have been better than what we got with the Khabibulin deal?
Our high-powered execs did not understand their bargaining position and wasted cap space on what Tyler has detailed as a bad bet.
Based on how good Lowe and MacT looked after blowing all the cap space they had in 2005-06, is amazes me that they do not appreciated its value.
The question is, if Biron signed for $1.4 million to back up Roli on July 22, what would Biron have signed for to start for the Oil on July 15?
and the question then becomes, how often is martin biron unemployed on july 15? that’s why it’s a false dichotomy - you can’t assume that martin biron or a goalie of his ilk is going to be without a contract late in the game, it’s more likely that someone in the scott clemmensen/craig anderson mold is going to be without a contract.
and even if biron is without a contract, once the number of suitors on biron jumps from 1 to 2 or 3, biron’s likely going to get paid fair market value.
i also don’t know why it’s assumed that biron will back up roloson - it’s going to be at least a split, and probably with more playing time for biron, before he gets sent to detroit, washington, san jose, or wherever.
i agree that in general, the right strategy in free agency this year was to wait, and the teams that jumped right in look like fools. and i agree, again, that the khabibulin contract was foolish. unfortunately when you have executives that need a playoff berth or they’re out the door, these are the kinds of moves that result.
Tyler,
Here’s the data that Vic was looking at, 5v5 expected SVPCT over the last two seasons:
http://www.behindthenet.ca/2008/5_on_5_goalie_shot_quality.php?sort=7&mingp=20
It’s clear that goaltender skill variation is a bigger component of SVPCT than defense (probably 2-3x). But the apparent difference between the difficulty of shots faced by Biron and Khabibulin is so large that it’s difficult for me to say that Biron’s performance was equivalent over the past two seasons. (I don’t think there’s any evidence that Chicago’s scorers have the bias that New York’s do.)
I don’t disagree that Khabibulin got too many years given his age. And his downside risk is huge in 2009-10. But I am pretty much convinced that he played much better than Biron recently.
Aside: looking at the data, I wonder if Backstrom is actually a better goalie than Kiprusoff?
Its not a ‘false dichotomy’. I have assumed that Khabby would have been signed elsewhere (Chicago - unlikely, KHL - more likely). If both Biron and Khabby were available on July 15, how good a deal can the Oil get on either one? Adding other options to the outcome makes the Oilers’ brass look worse - not better.
It would be refreshing to know that the men in charge could take a measured, calculated approach to their responsibilities instead of moving too early to chase the first shiny thing that they see (Dave Tippett for head coach anyone?)
It was obvious that there were more goalies than starter’s jobs. A quality goaltender was going to be left without a job. If the rumours about Biron’s salary demands are true, Biron determined that he would be the one.
It is the executives’ job to know their market. If the part-timers on the internet can count the potential suitors, why can’t the Oilers?
Clearly, the goaltending position that the Oilers gave Khabibulin is preferential to what Biron has in NY - now you are just being argumentative.
If the cap is goes down as much as speculated after this season, I doubt the wisdom of Biron’s move to gamble for a better market next off-season.
Gabe,
I used the data from NHL.com, EV only, to illustrate the pointlessness of the “shot quality” notion. EVsave% is, overwhelmingly, independent of team.
“How do you suggest that it be done? I’m as aware of problems with the methodology as anyone is but it seems to me that it is, at the very least, valuable to know this sort of stuff. What do you think that the Oilers are doing to project that he’ll be useful four years from now?” - Tyler
1- I suggest if you are using a inductive/comparison model, to use it to provide comparables (as LT does) rather projections. By giving multiple comparables, we can see two or three possible roads that a player might travel down, oppose to a pseudo-scientific, magic-ball, projection. Going back to my history example, orthodox Marxist induction science (in history or social planning) has been widely consider a failure because of its deterministic understanding of the future: that it takes patterns from induction and projects them as a absolute, or in some case a very high odds, projection of the future behaviours of individuals and specifically classes. Non-orthodox Marxist (as well as post-modern, and neo-liberal empiricism) history. all see patterns of experience or historical events as possible future paths but not in deterministic way: say for example in Honduras, the situation could unfold along multiple pathways and we can provide historical examples of the possibilities (using 1991 or 2004 Haiti, 1983 Guatemala, etc), opposed to using a inductive projection which may claim that the coup is a stage in a series of historical events towards a finality of history.
2- If you wanted to be more scientific then I would use deduction. Start with a hypothesis that your are going out to disprove (yes, in a negative methodology). Say start with the fact the Bulin CAN be a good goalie for the next four years, and then attempt to disprove this point (say looking at the mass of data you have and examine if any goal over 35 accomplishes the stats of want you consider to be a 3.5 million player). If the answer is no every time (yes at 100% of the time) you can say for sure that Khabibulin will never be worth his contract, and if it is not, you will eventually be able to come up with a percentage of the number of players that have been able to meet the value of their contract at his age, through all four years of the contract. This would give you the odds, I asked for, and a stronger scientific case against the Bulin Wall.
3- I have no idea what the Oilers are doing to project Khabibulin for four years, but I do know what you are doing since you are kind enough to share. If the Oilers release their methodology and it was similarly flawed, I would attack the crap out of it: more so then what I tried to do with your interesting study, since they are professionals and I am assuming this is your hobby.
Tyler
You don’t get enough credit for the quality of your reasoning, nor do you get enough credit for your ability to translate that reasoning to math in a succinct and honest way.
Few here appreciate the full value of this model. It’s certainly something I wouldn’t have thought of doing. And for those that haven’t read the link to Tango’s discussion, I would highly recommend that they do.
Tom expands on the original model by by addressing the obvious question: “Is 5500 shots really enough of a sample to accurately gauge a goalie’s natural save% ability”. And he does so with a rational model, one that is reversible btw. Very simple, clever and wholly sensible.
And he is quite right that we could expand on this to determine the ability distribution of NHL goalies. Though his “better way of doing that” mentoned later in the thread probably isn’t better at all. I suspect that he is thinking of summing the variances and producing a binomial (essentially normal, i.e. guassian) distribution. Even iteratively solving for K in the equation of the beta form: sp^(Kn-1)*(1-sp)^(K(n-1)-1) … where n=league average sp … and even if he did implement this Bayesian approach, I highly doubt that he would be able to create an ability distribution as expressed by Ken Holland and interpretted by yourself at Tango’s forum. And therefore it wouldn’t carry the same predictive value.
And there is far more value than that, but I doubt that anyone is still reading, so I’ll cut myself off here.
In short:
Keep up the good work, it is appreciated by many. I, for one, feel privileged to have access to your dementia.
I would rank the four factors influencing save percentage in order of importance as:
1) Observed goaltender performance
2) True goaltender talent
3) Observed defensive performance
4) True defensive talent
There is also observation error, which particularly affects our assessment of #3. In most cases, over a large number of shots, I agree that #1 accounts for enough of the difference in SVPCT that we don’t need to worry about #2.
But the difference between #3 for Biron (914) and Khabibulin (899) over 2400 and 1600 shots, respectively, is large enough that I don’t think we can ignore it, especially given that their observed 5v5 SVPCT is identical (916 B vs 918 K).
I should go back and run 2005-06 and 2006-07 if I can find the right data. It’s possible the situation was different in those years. But over the last two seasons, I would argue that Khabibulin likely performed better than Biron.
Gabe:
If I fished through agricultural microdata and came to the conclusion that, of 30 farmers in question, they averaged 40 acres of corn in the field. 1200 acres of corn total. And the amount of corn per farmer varied widely. How do you question that?
What if Tyler goes out and measures the total amount of land that these 30 farmers own, combined?
If that total is 90 acres, 3 acres per farmer, well reasonable people everywhere are going to start thinking that my estimate of 1200 acres of corn was based on flawed reasoning. Especially when we consider that there are good reasons to believe that other crop types are being grown.
I can question how well Tyler measured the land, but that’s hardly going to matter. Really, that 1100 acres of corn just can’t be explained.
If I argue long enough will I be able to convince Tyler to compromise on an estimate of 600 acres of corn in the fields? I doubt it, myself, but it might be worth a try.
I know that this whole Khabibulin thing has been beaten to death, and I think that almost everyone I read on the internet is of the same opinion, that being: term, age, and dollars considered … it’s clearly a bad bet.
But one other thing has crossed my mind. When Hossa balked on the Oilers offer a year ago he is reported to have been concerned about the Oilers goaltending. Now he may well have actually been thinking “if I’m going to sign long term, I want to know that this will be a good team. And that hot stretch the Oilers had at the end of last season may mean they’ve turned the corner, or it may have been just a bad team on a hot streak, like I’ve seen a bunch of times before. Including in Atlanta where I KNEW that we were, in fact, crap”.
That’s what I would have been thinking if I were him. But I don’t know. It was, however, widely reported that goaltending was the main issue that prevented him from signing on with the Oilers.
Knowing that the Oilers have a hard on for a star player, believing that this is the cure to all that ails them … is it completely mad to wonder if this signing of a famous goalie (Khabibulin) isn’t all part of that ill conceived master plan?
It’s important to note, as Ty has shown, that even if you believe that one goalie is better than the other, based on 8 save percentage points on 5000 shots, that this means you are only 58% sure that the guy who performed better is actually better.
For those who are making their arguments, are they accepting this finding?
I suggest if you are using a inductive/comparison model, to use it to provide comparables (as LT does) rather projections. By giving multiple comparables, we can see two or three possible roads that a player might travel down, oppose to a pseudo-scientific, magic-ball, projection.
I think that either you’re misreading what I’m saying or I’m not explaining it very clearly. In effect, what I have done is identify 247 comparators for the Biron/Khabby decision faced by the Oilers, defining my comparators by difference in save percentage and then tallying the results. I am not saying that since it’s usually a win for the goalie with the better save percentage, it’s always wrong to think that the guy with the lower save percentage is worse. I’m saying that, absent other information or circumstances, the odds favour the guy with the better save percentage. It seems to me that my analysis is explicitly premised on the idea that there are many different paths.
Going back to my history example, orthodox Marxist induction science (in history or social planning) has been widely consider a failure because of its deterministic understanding of the future: that it takes patterns from induction and projects them as a absolute, or in some case a very high odds, projection of the future behaviours of individuals and specifically classes. Non-orthodox Marxist (as well as post-modern, and neo-liberal empiricism) history. all see patterns of experience or historical events as possible future paths but not in deterministic way: say for example in Honduras, the situation could unfold along multiple pathways and we can provide historical examples of the possibilities (using 1991 or 2004 Haiti, 1983 Guatemala, etc), opposed to using a inductive projection which may claim that the coup is a stage in a series of historical events towards a finality of history.
Yeah, I’m a little confused by this, just because I think I’m pretty explicitly not a deterministic thinker. Shit, I thought that I even made it clear in this post, acknowledging the possibility of 60 games of .915 goaltending from him next year.
Basically, if I was running a team, I’d be pretty cognizant of the limitations of my own information and ability to perceive differences in players which, I think, kind of plays into your thinking. Not to turn this into a philosophical discussion but it seems to me that by proceeding the way that the Oilers have, they’ve basically stated that they think that they can predict the future of goalies with a high degree of certainty. Part of what makes this bet so bad is that they put so much money on it when there were other, cheaper bets available or the price of the bet could have been bargained down.
Gabe - I don’t trust the methodology. I’ve been down that road myself, coming up with a shot quality model. Yours seems to have some quirks that mine didn’t - the results I was getting when I did it made more sense to me although, IIRC, mine was more of an expected goals against thing that factored in shot volume - but I have difficulty understanding how Osgood is playing behind a terrible defensive team (in terms of shot quality) in Detroit while the guys in LA are enjoying the high life. It’s a smell test thing for me.
Say start with the fact the Bulin CAN be a good goalie for the next four years, and then attempt to disprove this point (say looking at the mass of data you have and examine if any goal over 35 accomplishes the stats of want you consider to be a 3.5 million player). If the answer is no every time (yes at 100% of the time) you can say for sure that Khabibulin will never be worth his contract, and if it is not, you will eventually be able to come up with a percentage of the number of players that have been able to meet the value of their contract at his age, through all four years of the contract. This would give you the odds, I asked for, and a stronger scientific case against the Bulin Wall.
To be clear: I think that Khabibulin may well prove to be worth his contract. I think that it’s an unknown and unknowable right now. I understand what you’re saying in terms of methodology but, as far as I’m concerned, I don’t need to go much further beyond showing that this isn’t a slam dunk and that there were options that were either better or at least of equivalent expected value available for less. The soft stuff - injuries and age - is just icing on that cake.
Tom said:
It’s important to note, as Ty has shown, that even if you believe that one goalie is better than the other, based on 8 save percentage points on 5000 shots, that this means you are only 58% sure that the guy who performed better is actually better.
For those who are making their arguments, are they accepting this finding?
Well, Tyler says that the higher save percentage group went 104 wins, 65 losses in their next 4 years & 2000+ shots (so 62% by my math). So it will be in that range.
I just ran your model, which assumes a .002 save percentage difference in actual ability between the two sets of players, and with normally distributed ability amongst the population.
Over a thousand trials it produces 40 fewer wins, 72 fewer losses, and 112 more ties than Tyler’s prorated data. That’s a lot of extra ties.
I don’t know why, though I think it’s a fair guess that nature hasn’t been so kind as to distribute the talent in binomial fashion. Also PPsave% contains a measurable element of team effect. The NHL doesn’t publish EVsave% data for the timeframe that Tyler was looking at, he could only use what was available (overall save%).
I think that the 58% comes if you treat it as a winning percentage.
Tyler,
Nice work. Oddly, I find the results kind of encouraging for the Oil going forward. (I would’ve expected a worse W/L ratio.) Hopefully Khabibulin is one of the 65 ‘wins,’ or that he and Biron both do well and Khabi is one of your ‘78 ties.’
I think trying to look at, not just ESV% from year to year, but entrenched ESV% taken over a number of years, was a good idea.
I think LT will pop up soon to tell you that the age specific numbers shouldn’t be counted because goalies last longer these days.
I suppose I’m not interested in comparing Khabibulin to Biron so much. I’m more interested in the odds that Khabibulins ESV% stays high after this past good year and the three previous -to coin a phrase- ‘vomitations.’
I mean ‘65 losses’
nature hasn’t been so kind as to distribute the talent in binomial fashion
Talent behaves in a Poisson fashion. That is certainly how the results show up - points for skaters, save % for goalies.
mc79hockey: Thanks for trying to explain your project better . . . I get it and think it has some value, but still have some of the same issues. I just think it is disingenuous to begin with a statement about objective knowledge/science vs. activism/opinion, and go on using a methodology that is primary an inductive opinion. I do not think you are deterministic, but I have reservations about the way your study is carried out (that it seems that an a priori value was established on Bulin and Biron, and then a historical analysis was carried out to objectively provide evidence for the opinion.)
Sorry for being a jerk, and I am glad to see such highly intelligent studies being performed on hockey. I like our work but do like your methodology.
“…our current federal government, which seems to govern on the basis that the only thing more important than actually solving a problem is being seen to have solved a problem by a sufficiently broad segment of the electorate that you can ensure your re-election.”
Pandering while using a minimum of “offensive” conservative policy, to get the majority… followed by 4 yrs of hard-right policy.
“I’m a nominal Tory who is now thinking awfully hard about switching allegiances. I really don’t like the American conservative movement and the Tories seem to me to be on that course.”
I’m not sure how anyone could be surprised by the Harper Con gov’t & it’s ties to American neo-conservatism, considering the Reform/R.W. roots. Remember Harper demanding from the Lib gov’t that Canada stand “shoulder to shoulder” with the Americans in Iraq? Remember Harper speaking to an American conservative movement & denigrating Canada before them?