• Spanish finance students, airport security and Corey Crawford

    by  • September 3, 2013 • Hockey • 17 Comments

    A little while back, a blog post by a guy named Chris Dillow made the rounds of various people’s Twitter feeds that I follow:

    Jordi Brandts and colleagues got a group of students to predict a sequence of five coin tosses, and then selected the best and the worst predictor. They then asked other subjects to bet on whether the best and worst predictor could predict another five coin tosses. The subjects were told that they would bet on the worst predictor from the first round, unless they paid to switch to the best predictor.

    82% of subjects paid to make the switch.

    But of course, there is no such thing as an ability to predict the toss of a coin. Most subjects, then, saw skill where there was only luck. And, what’s more, they were willing to spend good money to back this daft opinion.

    These people weren’t just idiots plucked from the street. They were fourth year finance undergraduates at one of the best universities in Spain.

    It’s been a while since I wrote about the NHL’s perpetually incoherent goaltending market and the Corey Crawford deal provides a sort of nice jumping off point for it. Crawford, who is 28 years old, has one year remaining on his deal at $2.5MM per. In his NHL career, he’s played 152 games. He’s posted a .913 save percentage in this time on 4009 shots. Conveniently for him, he had a .926 save percentage in 2013 over 769 shots in 30 games. Oh, and he won a Stanley Cup. Chicago’s signed him to a deal for ages 30-35 which pays him $6MM per season. Big money.

    As I’ve documented over the years, I think that there’s a pretty good case to be made that NHL GMs and Spanish finance students have something in common, namely difficulty in sorting the signal from the noise, the luck from the talent, when it comes to paying players. This is a not uncommon problem – it exists in a lot of different facets of human existence. Most of those don’t matter too much though. We spend billions of dollars that could be more usefully spent to create a security theatre at airports? Other than the occasional moment during which a fellow is jamming his hand a little too far into your pants, it doesn’t really affect you. Your favourite hockey team is saddled with a bad goalie contract? That’ll wreck a winter.

    I thought that this would end up being a multi-part deal but Dave Lozo kind of made the point I was going to make about the goalie market for next year (as in the Khabibulin Summer, “Goalies > Goalie Jobs” – see this, from 11 months before he was signed) so I’m just going to talk a little bit about putting a price tag on goalies and what you can expect from a guy like Crawford over the life of that deal.

    Screen Shot 2013-09-03 at 2.15.27 PM

    You need to develop sensible comparables to talk about this stuff. The first thing to think about is what you might expect an average starter or backup to do. I created a pool of starters between 2005-06 and 2012-13 by defining the goalie who played the most games for each team as the starter and throwing all of the other goalies into a backup pool. I then summarized the data in the table at left.

    If you want to talk about starting goalies and their value, you need a baseline. I’m a fan of using replacement level thinking for this: a goalie’s value is the number of goals that he prevents above his freely available replacement. Fortunately, this is pretty easy with hockey goalies – the freely available replacement is sitting at the end of the bench, opening the gate. As you can see from the table above, the relationship between the performance of NHL starters and NHL backups is pretty similar from year to year – the starters have a save percentage over the past eight years that averages about 8 points better per season.

    If you do the math on this, assuming the average team faces 2,445 shots a year and the starting goalie faces 2/3 of them, you come out to a league average starting goalie preventing about 13 goals above replacement level (ie. league average backup goalie) a year. The math, for those interested is 2445*2/3*.008=~13. 13 goals prevented is worth about 2.17 wins or 4.33 standing points.

    What’s that worth? I did some thinking on marginal points a few years ago. Effectively, what I was interested in was the amount of money that teams spend per marginal point in the standings. So, if you paid an entire team the minimum salary, I’d expect them to accumulate 48 points for 23*minimum salary; what you spend above that should bring you results, otherwise you’re just lighting money on fire.

    The reasoning is explained in that post that I’ve linked but let’s do some back of the envelope math for the 2013-14 NHL season. We’ll assume that there will be 300 OT/SO games. That means that there’s a total of 1230*2+300=2760 points available. Of those, 2760-(48*30)=1320 are “marginal points” – we assume that any team could get 48 points with a roster of guys paid the minimum. Those 1320 points are the points that teams are competing (and spending) to get.

    The NHL’s in a bit of a goofy spot right now as it transitions to a new salary cap. That being said, the CBA defines the mid-point this year as being $54.15MM. We’ll assume that this is precisely the amount of money to which the players will be entitled this year – $54.15MM multiplied by 30 teams works out to $1.625B paid to the players. From this, you subtract the amount of money that teams are required to pay to get the marginal spending – so 30 teams, multiplied by $550K and then by 23 roster spots per team. That gives you $379.5MM. $1.625B – $379.5MM = $1.245B in marginal spending on players.

    To get a league average cost per marginal standings point then, it’s simple: divide the marginal spending by the marginal points available. If you do that, you come up with a figure of about $943K per standings point. This seems to be in the ballpark to me.

    Now, we established above that a league average starting goalie is worth about 2.17 wins or 4.33 standing points. $943K multiplied by 4.33 standing points comes out to $4.1MM in marginal dollars. If you add the league minimum salary to that ($550K), you come up with $4.65MM. So by that analysis, paying $6MM for a league average starting goalie seems like a bit of a bad buy.

    I have a little bit of a problem with that line of thinking – a league average backup doesn’t earn minimum wage and I think you need to allow for that if you’re using that as your replacement level for starting goaltenders. If he’s making $1MM marginal dollars (ie. $1.55MM on average for a league average backup), the number looks less bad. As well, unrestricted free agents tend to make more in the NHL than restricted free agents – the price per marginal standings point from a UFA would be higher than from an RFA, which would make $6MM for a league average starting goalie less problematic in terms of what such a thing would cost on the open market.

    There’s also the fact of the salary cap likely rising dramatically in the next couple of years. This is widely expected, with people like James Mirtle talking about the potential for an $80MM cap. As the cap rises, the price of standings point rises. I think you can make a reasonable argument that if a man knocked on your door and said “I can guarantee you a league average starting goalie for six years if you pay me $36MM over that time” you can make a case that it would be a sensible deal to make, although you might also reasonably think that you can get league average starting goaltending for less.

    That said, the real problem with this deal is that it was given to Corey Crawford. Crawford’s save percentage for his career is a thoroughly average .913 – the bulk of his work has been over the past three seasons, so he’s actually slightly below average in terms of save percentage relative to the NHL. I wanted to get a sense of how goalies age, so I ran a search at hockey-reference.com for goalies who a) had accumulated at least 100 games played between 1995-96 and 2006-07 and b) did so through their age 28 season. That produced a list of 38 goalies. Then I looked at how those goalies did up to age 28 and then from age 30 onwards. It’s not particularly pretty.

    Of the 38, fully 22 of them faced fewer than 3200 shots from age 30-35, which is roughly equivalent with two years work for a starting goalie. As a group, these guys had a .904 save percentage in their pre-28 years, which isn’t bad when you consider that we’re talking about guys who played 10-15 years ago. They were, collectively, a complete disaster between 30-35.

    Screen Shot 2013-09-03 at 8.31.49 PM

    If Crawford posts these sorts of numbers in terms of appearances or save percentage decline, this move is a total disaster for Chicago. It’s not at all unthinkable, when you look at the number of guys in there who were name brand guys through age 28 and were then finished pretty quickly after the odometer clicked into the thirties.

    That leaves 16 qualifying guys who managed to crank out more than 3200 SA between 30-35. Here’s the list.

    Screen Shot 2013-09-03 at 11.07.37 PM

    Some things that stick out: not many guys are providing six years worth of starter level goaltending at that age. Outside of Tomas Vokoun, nobody’s getting dramatically better. Most guys post a worse save percentage from 30-35 than they did up until age 28 – and keep in mind, I’ve already filtered out the 22 guys who were so bad or so hurt that they couldn’t even generate two starter level years worth of shots against.

    So if the question is, is Corey Crawford likely to be a league average goalie between 2014 and 2020, it’s hard to conclude that the answer is yes. He hasn’t really been one to date in his career and goalies don’t seem to age well between the ages of 30 and 35. Why, then, would the Blackhawks do a deal like this?

    Well, one answer is that they think that they have information that Crawford’s a good bet despite the historical data. I assume that’s what they think; whether it’s actually true is another issue. Miserable goaltending is awful. It’s awful as a hockey fan and I bet it’s even more awful if you’re working for a hockey team. You want to protect against it. If you have a goalie who has a great year, you want to believe that it can be repeated. It’s not nearly as clear that it’s chance rather than skill as it is with the coin flipping that fooled Barcelona finance students who should have known better. A goalie making saves looks great. There’s probably an enormous temptation to try and protect yourself against bad goaltending by spending big resources on a guy who looks like a sure thing.

    The whole thing kind of reminds me of the security theatre issue in airports.

    Since 9/11, the U.S. has spent more than $1.1 trillion on homeland security.

    To a large number of security analysts, this expenditure makes no sense. The vast cost is not worth the infinitesimal benefit. Not only has the actual threat from terror been exaggerated, they say, but the great bulk of the post-9/11 measures to contain it are little more than what Schneier mocks as “security theater”: actions that accomplish nothing but are designed to make the government look like it is on the job. In fact, the continuing expenditure on security may actually have made the United States less safe.

    On the basis of the analysis here, goalies don’t tend to age that well in recent history. We are reasonably confident that one good year from a goalie tells us pretty much nothing about him – the list of guys who posted a .925 or better in a season in which they played 25 or more games includes such immortals as Vesa Toskala, Byron Dafoe, Andrew Raycroft, Ron Tugnutt, Marty Turco, Jose Theodore, Chris Mason, Ryan Miller, Manny Legace, Cristobal Huet, Brian Elliott, Roman Cechmanek and Sergei Bobrovsky. We know that unexceptional goalies have exceptional years.

    I suspect that teams want to believe that the great goaltending season happened because of a great goalie. I’m sure that they want to ensure that they keep a great goaltender. It just seems to me that, if we know that most goalies flame out sometime in their early thirties and we know that single seasons don’t tell us that much about a goalie, it’s a pretty hellacious risk to take. It’s a vast cost for a questionable benefit that may actually have made the Chicago Blackhawks less safe in net, in that they’ve made a big financial commitment to god knows what.

    The really scary thing, if you’re a Blackhawks fan, is the knowledge that big contracts for goalies in their thirties is a relatively new thing, particularly handing them out so far in advance. I suspect that part of the reason we see so many goalies just disappear from the NHL at this stage is that there wasn’t a contract that gave a team some incentive to just keep sending the player back between the pipes. One of the ugliest examples on my list is Marty Turco who was a) awful and b) played a lot. Turco signed a four year deal in his age 30 season that paid him $5.7MM a year. So he played. The uncomfortable thing for Chicago will be if Crawford suffers a Turcoesque decline – all of a sudden you have a backup quality goaltender making $6MM for six years.

    It seems like a bad risk to me, a heck of a curious thing to bet on, given what’s happened historically. Then again, if finance students can’t figure out that predicting coin flips is luck and the American government doesn’t realize how pointless a lot of the security theatre is, what hope does an NHL GM faced with a goalie who just posted a .926 save percentage and lifted the Stanley Cup have?

    Email Tyler Dellow at tyler@mc79hockey.com


    17 Responses to Spanish finance students, airport security and Corey Crawford

    1. speeds
      September 3, 2013 at

      CHI sure seems to have a history with these kinds of deals:

      Khabibulin – 4 years at 6.75M per, age 32-35
      Huet – 4 years at 5.625M per, age 33-36

      Khabibulin’s deal didn’t seem to work out that well for CHI; they signed Huet 3 years into Khabibulin’s 4 year deal. Huet spent the last 2 years on his deal outside the NHL. Perhaps the third time will be the charm?

      The other thing with a deal like this, going forward, is that a team can’t bury it in the minors like this sort of deal could be in the previous CBA.

      • Triumph
        September 4, 2013 at

        The Huet contract wasn’t that bad in theory (well, the $$ was, but the term wasn’t awful), and then the economy crashed two weeks later and the cap stayed put the next season. Still, he had had 3 seasons where he was one of the best goalies in the league leading up to that one.

        As for not being able to bury it, that’s true, but they can retain salary on it. Corey Crawford will always be a Cup-winning goalie and that carries enough cachet with the dumber teams that the contract would move if it had to (unless he was a total disaster).

        Also, Tyler’s chart above is not era-normalized – save percentage went down significantly in the 2005-06 and 2006-07 seasons. That isn’t to say this isn’t a bad bet, but it appears to be a worse bet than it actually is.

    2. September 4, 2013 at

      Goalies Don’t Improve.

      • Tyler Dellow
        September 4, 2013 at

        I screwed up – where I say “100 games played between 1995-96 and 2006-07″, the reference to 2006-07 should be 2005-06. I cut out anyone who wasn’t 28 yet either. If you sort your first link by year, you’ll see 11 guys who weren’t yet 28 by the end of the 2005-06 season – those guys got sliced out.

    3. John Gilbertson
      September 4, 2013 at

      There is a clear feedback loop between defensive play and goalie performance, just like there is with baseball defense and pitching performance. When the Blackhawks had strong D play – 2011 and 2013 – Crawford (“Coco”) played extremely well. When the D play slipped – 2012, after trade of Brian Campbell – Coco’s play slipped too. This is true for all but 1 or 2 goalies in the world (maybe – maybe! – Lundqvist). Well, the Blackhawks have best-in-class D play and will for a couple more years, maybe longer. They decided to keep Crawford, to press their advantage, and this is the market rate. They may get burned on the back end of the contract, when Coco is age 32+ and the team is past the current peak, but so what? As noted, that happened before, and they still won the Cup in 2010.

      • Andy Dancer
        September 5, 2013 at

        I don’t think defensive play has as much an impact on goalie performance as you may be suggesting. The correlation between sv% and shot distance, for instance, is very weak on the team level: http://www.hockeybuzz.com/blog.php?post_id=52139#.Uijirz_dR3k

      • Andy Dancer
        September 5, 2013 at

        That is, unless you are evaluating goalie performance using, say, GAA.

    4. Henry
      September 4, 2013 at

      Chicago just extended Hjallmerson too at $4.1M for five years. Something’s going to have to break because they have $60M tied into 14 contracts for next year already. Could be an interesting deadline for them.

      • Triumph
        September 4, 2013 at

        If the cap is at $68-70M like many expect it to be, I don’t see why this is the case. They don’t have any critical players with expiring contracts. Even so, I could see them moving Oduya, Hjalmarsson, Leddy, or Roszival depending on how their defense prospects shape up.

    5. September 4, 2013 at

      My quick take on this was similar. The opportunity cost of the deal is awful. But in terms of the expected performance, it is important to remember that Chicago is a superb shot-prevention team. If they can keep the ice firmly tilted away from Crawford’s goal over the life of this extension, they’ll likely be fairly happy with his performance. Which, again, doesn’t really constitute an argument FOR making him one of the highest-paid goalies in the NHL.

      • John Gilbertson
        September 4, 2013 at

        The issue here is not about price. Crawford got the market rate for a guy who just won the Jennings, the Cup, and was a near miss for Conn Smythe (was also the all-Rookie goaltender in 2011). The real decision was whether to let him go in 2014, and trust the new Antti and the market for cheaper free agents, as they did with the old Antti. The Blackhawks are serious Cup contenders for the next 2 years, probably longer, which is exactly when you can tolerate some risk of overpayment. On this team, an equal risk was Coco having another strong season, pushing his price above 6*6 (Rask got 8*7; is he any better?). Stan can afford to pay the current market, and decided not to temp fate. “Awful” seems like alarmist exaggeration of the Blackhawks’ risk in making this deal.

        • dawgbone
          September 6, 2013 at

          The risk of overpayment isn’t the next 2 years. If this deal was 2 years @ $6mil no one would bat an eye.

          The deal is 6 years and takes him to 35 years old and, career year aside, Crawford’s been a pretty average goaltender and goalies tend not to improve on the other side of 30 (the Roloson’s and Thomas’s aside).

    6. Bill
      September 4, 2013 at

      Crawford is a Top 50 player in the NHL. Hockey News says so. Same with Price. These guys are golden. Don’t sweat it

    7. Pingback: Spectors Hockey | NHL Blog Beat – September 5, 2013.

    8. September 5, 2013 at

      Re: your discussion near the end as to why GM’s do this particularly with goaltenders, I looked at the possibility that “probability neglect” may be driving less than rational decision making back in 2011:


    9. Pingback: Eakpectations | oilersnow

    Leave a Reply

    Your email address will not be published. Required fields are marked *