A little while back, a blog post by a guy named Chris Dillow made the rounds of various people’s Twitter feeds that I follow:
Jordi Brandts and colleagues got a group of students to predict a sequence of five coin tosses, and then selected the best and the worst predictor. They then asked other subjects to bet on whether the best and worst predictor could predict another five coin tosses. The subjects were told that they would bet on the worst predictor from the first round, unless they paid to switch to the best predictor.
82% of subjects paid to make the switch.
But of course, there is no such thing as an ability to predict the toss of a coin. Most subjects, then, saw skill where there was only luck. And, what’s more, they were willing to spend good money to back this daft opinion.
These people weren’t just idiots plucked from the street. They were fourth year finance undergraduates at one of the best universities in Spain.
It’s been a while since I wrote about the NHL’s perpetually incoherent goaltending market and the Corey Crawford deal provides a sort of nice jumping off point for it. Crawford, who is 28 years old, has one year remaining on his deal at $2.5MM per. In his NHL career, he’s played 152 games. He’s posted a .913 save percentage in this time on 4009 shots. Conveniently for him, he had a .926 save percentage in 2013 over 769 shots in 30 games. Oh, and he won a Stanley Cup. Chicago’s signed him to a deal for ages 30-35 which pays him $6MM per season. Big money.
As I’ve documented over the years, I think that there’s a pretty good case to be made that NHL GMs and Spanish finance students have something in common, namely difficulty in sorting the signal from the noise, the luck from the talent, when it comes to paying players. This is a not uncommon problem – it exists in a lot of different facets of human existence. Most of those don’t matter too much though. We spend billions of dollars that could be more usefully spent to create a security theatre at airports? Other than the occasional moment during which a fellow is jamming his hand a little too far into your pants, it doesn’t really affect you. Your favourite hockey team is saddled with a bad goalie contract? That’ll wreck a winter.
I thought that this would end up being a multi-part deal but Dave Lozo kind of made the point I was going to make about the goalie market for next year (as in the Khabibulin Summer, “Goalies > Goalie Jobs” – see this, from 11 months before he was signed) so I’m just going to talk a little bit about putting a price tag on goalies and what you can expect from a guy like Crawford over the life of that deal.
You need to develop sensible comparables to talk about this stuff. The first thing to think about is what you might expect an average starter or backup to do. I created a pool of starters between 2005-06 and 2012-13 by defining the goalie who played the most games for each team as the starter and throwing all of the other goalies into a backup pool. I then summarized the data in the table at left.
If you want to talk about starting goalies and their value, you need a baseline. I’m a fan of using replacement level thinking for this: a goalie’s value is the number of goals that he prevents above his freely available replacement. Fortunately, this is pretty easy with hockey goalies – the freely available replacement is sitting at the end of the bench, opening the gate. As you can see from the table above, the relationship between the performance of NHL starters and NHL backups is pretty similar from year to year – the starters have a save percentage over the past eight years that averages about 8 points better per season.
If you do the math on this, assuming the average team faces 2,445 shots a year and the starting goalie faces 2/3 of them, you come out to a league average starting goalie preventing about 13 goals above replacement level (ie. league average backup goalie) a year. The math, for those interested is 2445*2/3*.008=~13. 13 goals prevented is worth about 2.17 wins or 4.33 standing points.
What’s that worth? I did some thinking on marginal points a few years ago. Effectively, what I was interested in was the amount of money that teams spend per marginal point in the standings. So, if you paid an entire team the minimum salary, I’d expect them to accumulate 48 points for 23*minimum salary; what you spend above that should bring you results, otherwise you’re just lighting money on fire.
The reasoning is explained in that post that I’ve linked but let’s do some back of the envelope math for the 2013-14 NHL season. We’ll assume that there will be 300 OT/SO games. That means that there’s a total of 1230*2+300=2760 points available. Of those, 2760-(48*30)=1320 are “marginal points” – we assume that any team could get 48 points with a roster of guys paid the minimum. Those 1320 points are the points that teams are competing (and spending) to get.
The NHL’s in a bit of a goofy spot right now as it transitions to a new salary cap. That being said, the CBA defines the mid-point this year as being $54.15MM. We’ll assume that this is precisely the amount of money to which the players will be entitled this year – $54.15MM multiplied by 30 teams works out to $1.625B paid to the players. From this, you subtract the amount of money that teams are required to pay to get the marginal spending – so 30 teams, multiplied by $550K and then by 23 roster spots per team. That gives you $379.5MM. $1.625B – $379.5MM = $1.245B in marginal spending on players.
To get a league average cost per marginal standings point then, it’s simple: divide the marginal spending by the marginal points available. If you do that, you come up with a figure of about $943K per standings point. This seems to be in the ballpark to me.
Now, we established above that a league average starting goalie is worth about 2.17 wins or 4.33 standing points. $943K multiplied by 4.33 standing points comes out to $4.1MM in marginal dollars. If you add the league minimum salary to that ($550K), you come up with $4.65MM. So by that analysis, paying $6MM for a league average starting goalie seems like a bit of a bad buy.
I have a little bit of a problem with that line of thinking – a league average backup doesn’t earn minimum wage and I think you need to allow for that if you’re using that as your replacement level for starting goaltenders. If he’s making $1MM marginal dollars (ie. $1.55MM on average for a league average backup), the number looks less bad. As well, unrestricted free agents tend to make more in the NHL than restricted free agents – the price per marginal standings point from a UFA would be higher than from an RFA, which would make $6MM for a league average starting goalie less problematic in terms of what such a thing would cost on the open market.
There’s also the fact of the salary cap likely rising dramatically in the next couple of years. This is widely expected, with people like James Mirtle talking about the potential for an $80MM cap. As the cap rises, the price of standings point rises. I think you can make a reasonable argument that if a man knocked on your door and said “I can guarantee you a league average starting goalie for six years if you pay me $36MM over that time” you can make a case that it would be a sensible deal to make, although you might also reasonably think that you can get league average starting goaltending for less.
That said, the real problem with this deal is that it was given to Corey Crawford. Crawford’s save percentage for his career is a thoroughly average .913 – the bulk of his work has been over the past three seasons, so he’s actually slightly below average in terms of save percentage relative to the NHL. I wanted to get a sense of how goalies age, so I ran a search at hockey-reference.com for goalies who a) had accumulated at least 100 games played between 1995-96 and 2006-07 and b) did so through their age 28 season. That produced a list of 38 goalies. Then I looked at how those goalies did up to age 28 and then from age 30 onwards. It’s not particularly pretty.
Of the 38, fully 22 of them faced fewer than 3200 shots from age 30-35, which is roughly equivalent with two years work for a starting goalie. As a group, these guys had a .904 save percentage in their pre-28 years, which isn’t bad when you consider that we’re talking about guys who played 10-15 years ago. They were, collectively, a complete disaster between 30-35.
If Crawford posts these sorts of numbers in terms of appearances or save percentage decline, this move is a total disaster for Chicago. It’s not at all unthinkable, when you look at the number of guys in there who were name brand guys through age 28 and were then finished pretty quickly after the odometer clicked into the thirties.
That leaves 16 qualifying guys who managed to crank out more than 3200 SA between 30-35. Here’s the list.
Some things that stick out: not many guys are providing six years worth of starter level goaltending at that age. Outside of Tomas Vokoun, nobody’s getting dramatically better. Most guys post a worse save percentage from 30-35 than they did up until age 28 – and keep in mind, I’ve already filtered out the 22 guys who were so bad or so hurt that they couldn’t even generate two starter level years worth of shots against.
So if the question is, is Corey Crawford likely to be a league average goalie between 2014 and 2020, it’s hard to conclude that the answer is yes. He hasn’t really been one to date in his career and goalies don’t seem to age well between the ages of 30 and 35. Why, then, would the Blackhawks do a deal like this?
Well, one answer is that they think that they have information that Crawford’s a good bet despite the historical data. I assume that’s what they think; whether it’s actually true is another issue. Miserable goaltending is awful. It’s awful as a hockey fan and I bet it’s even more awful if you’re working for a hockey team. You want to protect against it. If you have a goalie who has a great year, you want to believe that it can be repeated. It’s not nearly as clear that it’s chance rather than skill as it is with the coin flipping that fooled Barcelona finance students who should have known better. A goalie making saves looks great. There’s probably an enormous temptation to try and protect yourself against bad goaltending by spending big resources on a guy who looks like a sure thing.
The whole thing kind of reminds me of the security theatre issue in airports.
Since 9/11, the U.S. has spent more than $1.1 trillion on homeland security.
To a large number of security analysts, this expenditure makes no sense. The vast cost is not worth the infinitesimal benefit. Not only has the actual threat from terror been exaggerated, they say, but the great bulk of the post-9/11 measures to contain it are little more than what Schneier mocks as “security theater”: actions that accomplish nothing but are designed to make the government look like it is on the job. In fact, the continuing expenditure on security may actually have made the United States less safe.
On the basis of the analysis here, goalies don’t tend to age that well in recent history. We are reasonably confident that one good year from a goalie tells us pretty much nothing about him – the list of guys who posted a .925 or better in a season in which they played 25 or more games includes such immortals as Vesa Toskala, Byron Dafoe, Andrew Raycroft, Ron Tugnutt, Marty Turco, Jose Theodore, Chris Mason, Ryan Miller, Manny Legace, Cristobal Huet, Brian Elliott, Roman Cechmanek and Sergei Bobrovsky. We know that unexceptional goalies have exceptional years.
I suspect that teams want to believe that the great goaltending season happened because of a great goalie. I’m sure that they want to ensure that they keep a great goaltender. It just seems to me that, if we know that most goalies flame out sometime in their early thirties and we know that single seasons don’t tell us that much about a goalie, it’s a pretty hellacious risk to take. It’s a vast cost for a questionable benefit that may actually have made the Chicago Blackhawks less safe in net, in that they’ve made a big financial commitment to god knows what.
The really scary thing, if you’re a Blackhawks fan, is the knowledge that big contracts for goalies in their thirties is a relatively new thing, particularly handing them out so far in advance. I suspect that part of the reason we see so many goalies just disappear from the NHL at this stage is that there wasn’t a contract that gave a team some incentive to just keep sending the player back between the pipes. One of the ugliest examples on my list is Marty Turco who was a) awful and b) played a lot. Turco signed a four year deal in his age 30 season that paid him $5.7MM a year. So he played. The uncomfortable thing for Chicago will be if Crawford suffers a Turcoesque decline – all of a sudden you have a backup quality goaltender making $6MM for six years.
It seems like a bad risk to me, a heck of a curious thing to bet on, given what’s happened historically. Then again, if finance students can’t figure out that predicting coin flips is luck and the American government doesn’t realize how pointless a lot of the security theatre is, what hope does an NHL GM faced with a goalie who just posted a .926 save percentage and lifted the Stanley Cup have?Email Tyler Dellow at email@example.com