• Going off the board in the draft

    by Tyler Dellow • July 19, 2010 • Uncategorized • 38 Comments

    I’m working on some other things right now, which is part of the reason that posting’s been light of late. Internet hockey content seems to be drying up at the moment – Friedman just shut down for the summer but not before providing us with some quotes from scouts who were forced to remain anonymous.

    As an aside – the use of anonymous quotes here is ridiculous. I don’t blame Friedman but none of the scouts who are quoted says anything the least bit controversial. The NHL should want people generating new hockey content over the summer and they should be doing what they can to faciliate it.

    Scout number one said, “Fans should use Central Scouting rankings as a guideline,” not the definitive list.

    (Several different scouts, all requesting anonymity, were contacted for this piece. Four of them will be quoted. To make it easier, each is given a number. I think I’m setting a record for most anonymous quotes ever. Unfortunately, that’s the way this business goes.)

    “Everybody looks at Central Scouting or Red Line or other reports,” says No. 2. “In the end it’s what your guys see … not what they see.”

    “(Central Scouting) focuses on physical ability – not mental ability, work ethic or character,” says No. 3. “It’s up to regional scouts to sort through rumours and innuendo. If something is said about a prospect, you need them to determine if this is real or a negative vendetta. That’s why these regional guys are so valuable.”

    “They don’t look at character, or what guys are like in the room or on the bench,” adds Scout No. 1. “They have a different set of criteria.”

    “Most guys sit across from the bench all the time so they can watch how a player reacts when the coach talks to him,” says No. 1.

    This draft did have some examples of that. John McFarland, ranked 15th, went 33rd to Florida. Kirill Kabanov, the red-flagged bad boy of this class, was taken 65th, 34 spots above his North American ranking.

    Meanwhile, Jeff Skinner went seventh overall to Carolina, “Exactly where I would have taken him,” according to Scout 1. Skinner’s Central ranking was 34. (That was an improvement from his mid-season position of 47.)

    Elliotte’s piece is great – I wouldn’t usually quote this much from some but there’s so much there that I don’t feel I’m stealing the soul of the article. Unlike some other Oiler flavoured blogs, I tend to be a bit skeptical of their wisdom. I’ve read Gare Joyce’s book in which he talks about scouting at great length and a lot of it just seems like modern day phrenology to me. I sort of think that any reasonably competent scouting operation is able to generally sort guys into roughly where they should be but that it’s really a crap shoot beyond that. All of these things that they look for to differentiate players from one another – Brian Burke apparently liked Ryan Kesler because he put his arm around someone who was having a bad game – just strike me as so much skull reading.

    If the NHL as a whole barred scouting, doubled the budget of Central Scouting and forced teams to pick from the CSB list, I’m not convinced that the league would be appreciably worse at identifying players. I don’t think you’d suddenly see Ales Hemsky’s getting drafted at 200 or something like that. The bit in Friedman’s piece about Jeff Skinner caught my eye and, as it so happens, I have a couple of lists of CSB rankings handy. I thought I’d post the guys who were taken more than five spots ahead of where CSB had them in the first round in 2000, 2001 and 2004 for a bit of fun.

    Rick DiPietro, who went first in 2000 despite not being listed, is excluded – IIRC, he had to opt into the draft because he was in college or something, which is why he wasn’t listed. The list goes player, rank and pick; it’s sorted by first rounders who were picked the furthest from where CSB had them on their respective list. For example, Shaone Morrisonn was ranked 41st out of North American skaters in 2001, was the ninth NA skater picked and went 19th overall.

    Adrian Foster (NJD – 2001) – NR – NR – 28
    Shaone Morrisonn (BOS – 2001) – 41 (NA-S) – 9 – 19
    Kris Chucko (CGY – 2004) – 29 (NA-S) – 11 – 24
    Krys Kolanos (PHX – 2000) – 27 (NA-S) – 9 – 19
    Jens Karlsson (LAK – 2001) – 26 (Euro-S) – 8 – 18
    David Hale (NJD – 2000) – 25 (NA-S) – 10 – 22
    Niklas Kronwall (DET – 2000) – 26 (Euro-S) – 12 – 29
    Chuck Kobasew (CGY – 2001) – 20 (NA-S) – 6 – 14
    Blake Wheeler (PHX – 2004) – 17 (NA-S) – 3 – 5
    Marcel Hossa (MTL – 2000) – 20 (NA-S) – 7 – 16
    Alexei Mikhnov (EDM – 2000) – 19 (Euro-S) – 8 – 17
    Jason Bacashihua (DAL – 2001) – 12 (NA-G) – 3 – 26
    Lars Jonsson (BOS – 2000) – 11 (Euro-S) – 2 – 7
    Andy Rogers (TBL – 2004) – 24 (NA-S) – 16 – 30
    Artem Kryukov (BUF – 2000) – 14 (Euro-S) – 7 – 15
    Travis Zajac (NJD – 2004) – 15 (NA-S) – 9 – 20

    With a few exceptions, that strikes me as a pretty terrible collection of first round picks. If I were to add 2002 and 2003, Edmonton would put another overdraft bust on the list, in the persons of Jesse Niinimaki (50th on the Euro list, he was the 3rd Euro skater taken, at 15th overall).

    Only five first rounders fell more than five spots on their list – you’ve probably heard of two of them in Brad Boyes and Mike Green.

    It seems to me that if you own an NHL team, your scouting department ought to justify its existence by producing more than just different information from the information produced by CSB – it ought to be producing better information. This is just a taste of what I’m accumulating but, at first impression, I’m not inclined to think that when draft departments produce vastly different information from CSB that it’s better information.

    About Tyler Dellow

    38 Responses to Going off the board in the draft

    1. Devin
      July 19, 2010 at

      Tyler, been wondering about this very thing for awhile – interested to see your broader results. If a GM just took a look over the past say 10 years at RLR, CSB, etc and gave each list appropriate weight based on performance, and let “the machine” decide when their pick was up, the results would likely be far more consistent.

      Scouting seems to be such a black magic kind of thing for these teams – are they really getting any special value? Maybe if they have guys trolling around the Europe div ii leagues looking at 16 year olds that nobody knows… but even then?

      It strikes me that over time you’ve done a great deal to dismantle the notion that NHL GMs have a complicated job. There’s so much information out there about players and performance that it always amazes me when these guys make their decisions seemingly in a bubble.

    2. E
      July 19, 2010 at

      i am quite literally longing- nay, yearning- for the follow-up to this. the question has been nagging at me for like three years: is anyone actually good at scouting? is anyone actually able to pick consistently better than central scouting? the problem i’ve always run into is that most scouting evaluations are done by team, and a team is a hive mind comprised of changing individual components, and the individual components- actual scouts- are hard to track. so how to discern if there is such a think as individual expertise/insight? it drives me to distraction, this question.

    3. July 19, 2010 at

      I took a look at the same issue from a different angle. To sum up:

      Another reason investigating personal factors may be a concern in scouting is human perception isn’t great at filtering the signal from the noise. To put it another way – having a mountain of info is not necessarily a good thing. Not just because the relevant data may be merely lost amongst the flotsam like a needle in a haystack, but because potentially unrelated bits of info can actually influence evaluation. In psychological circles, this is called “the dilution effect” and it’s defined as the tendency for neutral or irrelevant info to weaken a judgment.

    4. David Staples
      July 19, 2010 at

      Just curious, if you were running an NHL scouting department today, how would you go about doing it? How would you make your evaluations?

      Me, I’d hire a team of interns and track true plus/minus off video of the major junior leagues, and do as much as I could of that in Europe as well.

      But how would you evaluate 17-year-old players?

    5. lowetide
      July 19, 2010 at

      DePietro was ranked #3 overall in the Hockey News draft issue (behind Heatley and Gaborik).

      David Staples: Not my question, but I’d run all the stats through Desjardins NHLE (which would boost people like Granlund and Skinner), nick the defensemen about 15 slots (unless they’re Potvin) and never draft a goalie before no. 100 overall.

      AND I’d refuse to draft anyone who had a significant injury (concussion, knee, etc) before draft day.

      Also, speeds nailed this year’s draft on his blog. Someone should hire him.

    6. Deano
      July 19, 2010 at

      Devin and Staples:

      How about this for ‘the machine’:

      Use a ‘normal’ scouting approach, and then draft off Bob Mackenzie’s’ list. (You keep scouts to help Bob make his list.)

      The Oilers were rumored to be trying to buy their way up to #15 to draft McIlrath (#17 NA-S) who was taken @ #10 (8th NA-S) by Slats.

    7. Tyler Dellow
      July 19, 2010 at

      LT – That means DiPietro was almost certainly #1 on the NA goalie list then.

    8. Deano
      July 19, 2010 at
    9. lowetide
      July 19, 2010 at

      That Sports Illustrated article says DiPietro was #1 ranked NA golaie, but according to the HN Draft Issue Brent Krahn was the #1 NA goalie that year. DiPietro isn’t listed.

    10. July 19, 2010 at

      I’d run all the stats through Desjardins NHLE (which would boost people like Granlund and Skinner), nick the defensemen about 15 slots (unless they’re Potvin) and never draft a goalie before no. 100 overall.

      I think this is exactly right.

    11. Vic Ferrari
      July 19, 2010 at

      I heard a guy from CSB on he radio a couple of monhs ago, I forge his name. In any case, he talked about the psychological tests that CSB runs on he top prospects, and it was exensive.

      So while CSB surely has warts, ignoring personal traits is no one of hem. In fact it is probably a strengh. I mean I personally would value he insighs of a pyschologist over the warm, fuzzy feeling of a team scout.

    12. Vic Ferrari
      July 19, 2010 at

      Fricken ‘t’s. I have to remember to lean on that key. That’s it, I’m tearing this computer apart.

    13. Tyler Dellow
      July 19, 2010 at

      I think that Kent’s point, about the dilution effect, is absolutely true. I’ll bet a lot of the guys who I’ve got tagged as overdrafts interview well, which sort of goes to Vic’s point as well. There’s a good essay in Malcolm Gladwell’s book “What the Dog Saw”, which is also available here, that I’ll quote from:

      This interviewing technique is known as “structured interviewing,” and in studies by industrial psychologists it has been shown to be the only kind of interviewing that has any success at all in predicting performance in the workplace. In the structured interviews, the format is fairly rigid. Each applicant is treated in precisely the same manner. The questions are scripted. The interviewers are carefully trained, and each applicant is rated on a series of predetermined scales.

      Not surprisingly, interview specialists have found it extraordinarily difficult to persuade most employers to adopt the structured interview. It just doesn’t feel right. For most of us, hiring someone is essentially a romantic process, in which the job interview functions as a desexualized version of a date. We are looking for someone with whom we have a certain chemistry, even if the coupling that results ends in tears and the pursuer and the pursued turn out to have nothing in common. We want the unlimited promise of a love affair. The structured interview, by contrast, seems to offer only the dry logic and practicality of an arranged marriage.

      I note that, in selecting between Seguin and Hall, Katz invited them over for dinner to meet the family. Do the Oilers involve an industrial psychologist in this? Are there even going to be substantiated methods of finding athletes who will be successful through interviews, given the relatively small number of elite athletes? I suspect that the answer to both questions is no.

    14. mclea
      July 19, 2010 at

      Ilya Kovalchuk’s contract is awesome(h/t @FriedmanHNIC):

      http://bit.ly/9S4ysC

    15. noah
      July 19, 2010 at

      I don’t have a link offhand, but the reference to Gladwell reminded me of it – was listening to an interview on CBC a month or three back about similar ideas, and apparently the only institution in the world that makes a conscious effort to avoid this type of “intuition” bias is the Israeli Air Force. They hire everyone, including all the people they tag in interviews as those who are not suitable for the IAF; after they go through the training etc either they flame out or succeed, and they take the whatever small percentage succeeds and re-interview/debrief to find out why they thought those people would fail and why the initial interviewers were wrong.

      Interesting stuff; understandably consumes a considerable amount of resources but makes a certain amount of sense in gaining a better understanding of how to actually approach your interviews.

    16. Vic Ferrari
      July 19, 2010 at

      noah,

      That’s very cool.

      I always wonder about the ability of scouts, or anyone for that matter, to separate head from heart.

      A long time ago, in the early 90s, I lived in England for a while. I had a great circle of friends. One of them was a Cypriot girl, her father was an abusive alcoholic who worked at Heathrow Airport. She grew up in London, Hounslow I think. She was also instantly likable, but batshit crazy once you got to know her. Still a very very cool girl, but crazy, no two ways about it.

      Any road, she was finishing a degree in something unrelated, linguistics, when she applied for a job as a personal assistant to an executive at a Japanese firm in London. Toyota I think, but I wouldn’t swear to it.

      A couple of fridays later at the pub she’s telling us the story of her new boss. Specifically how they had her handwriting analyzed and their expert had declared her mentally instable. Dude had a made a laundry list of shit that he thought was wrong wih Nikki. And generally I don’t believe in that black magic, but he was spot on. Obviously the guy had hired her anyways, otherwise she would never have known. He said that there was just something about her that he liked.

      Thing is, as she’s telling the story, for the first time that I can remember she’s wearing a tight sweater. It was all I could do to listen … I had no idea that she had such fabulous titties. Every fiber of my being was focussed on not staring directly at them.

      I don’t know what she wore to the interview, but I had a guess at what that “something about her” was that got her the gig.

      I think it applies everywhere though. It sounds crude to call it the “great tits” theory of evidentiary exclusion. But it’s there, I think. “Sexy” means different things for a hockey prospect than it does for a young female assistant, but the principle reamins the same. Or so I think.

    17. Vic Ferrari
      July 19, 2010 at

      While I’m reminiscing, one guy in that circle has gone on to a odd sort of fame.

      I’ll call him Paul, but that’s not his real name. He’s a prominent business consultant now, specifically to the banking industry. He’s never actually run a business in his life, apparently nobody ever asks.

      It was funny at first, and he had great stories, when he’d tell us about standing behind one way glass and picking out the non-believers by the manner in which they laughed at the CEO’s jokes. I don’t know much about the banking industry, but obviously being highly paid and highly replaceable is not a good combination. Wacky stuff, but let’s be frank, nobody likes bankers anyways … so it’s good for a laugh.

      A real money spinner, and practical joke gold btw, was his series of awards (plaques and paper weights (the latter was the biggest seller, oddly enough) that contained transmitting devices. You can’t just buy those anywhere, the margins were terrific.

      I sometimes wonder what’s going on with those. Attics and basements no doubt, some still on office walls. Few throw those sorts of things away, I don’t know how long the batteries last. Could you drive down the eastern seaboard with a receiver and pick up personal conversations?

      He got dark after that. Clever I grant you. When you think about it, if you can think up shit like that you can think up worse, I suppose it was inevitable. I don’t think there was evil there, nothing of the sort. You get ino a business, you try to succeed, you have employees that depend on you … you just try to move forward, you try to move the business forward. Shit happens. I think he was trying to do the right thing on the day.

      Fuck it, I don’t like him any more, but I doubt he cares. If he does, he can go cry himself to sleep on a big bag of money.

    18. July 20, 2010 at

      Ryan O’Marra apparently destroyed the Islanders’ psychological assessments.

      And the correlation between NHL points and a weighted combination of draft day points, plus/minus and age is just as good for major junior players as the correlation between NHL points and draft position.

      I think the scouts do good work, on the whole, but the reality is that there’s very little reason for any team to believe they’re any better at it than anyone else.

    19. Tyler Dellow
      July 20, 2010 at

      David –

      I wanted to think through your question a little bit. There are a lot of points that are important in reasoning through it, I think.

      The first point is how we measure scouting success. Lowetide’s argued that the Oilers have drafted well in the last five years or so (I think I’m putting his position forward correctly; I’m sure he’ll correct me if I’m wrong). I’m not as sure that I’d agree.

      It seems to me that you want your draft department to generate as much surplus value for you as possible. When I say “surplus value”, I’m referring to the difference between what a player costs under his contract versus what the value of the wins that he would provide if they were purchased on the open market.

      For some players right now, I’m not sure that there can be any surplus value. Goalies, for example. Unless you get an absolute stud in the draft – I’m talking Luongo/Vokoun class – I’m not sure it makes sense to even draft a goalie right now because wins are so cheap to purchase from them on the open market. Fourth liners cost basically nothing as do third pairing D. A scouting department has to crank out an awful lot of third/fourth liners, averagish goalies and 5/6 D to cover the costs of the scouting department, I would think.

      Now, even if the scouting department can cover the investment that the team makes in it – say it’s generating $2MM in excess value annually on expenditures of $1.5MM – teams should still be looking for better returns on their investment where it’s possible. Say, for example, the Oilers could pay speeds $100K annually and generate $1MM in surplus value. They’re up $900K instead of $500K and have an additional $1.4MM available to invest in other areas.

      I think that there’s a lot of evidence to suggest that the returns on the marginal investment in scouting by individual teams are pretty lousy. CSS is effectively speeds but at a cost of zero marginal dollars.

      To come back to my point about Lowetide then, I’m not convinced that the Oilers scouting has found them a ton of players who have been on real value contracts in the past few years. Part of that is the team pissing away value years and maybe part of it is that the drafting has generally sucked but that 05-06 team had a lot of guys on value deals, many of whom were found by the scouts. Today’s team doesn’t.

      If I were running a team, therefore, I’d probably instruct my guys to look for guys who might be top sixers if things broke right. In that world, I’d have a good understanding of the cost of wins and the value (in wins) that a given player represents and I’d probably be strongly tilted in favour of players with top six potential over players who are sure things at the bottom of the lineup. I might consider taking runs pretty consistently at the higher risk, higher upside guys.

      I also think that there’s a pretty good argument to pay attention to guys who fall on the CSS list, particularly as you get deeper into the draft. Again, I’m working from a limited sample (00, 01 and 04) but in rounds 2/3, guys who fell more than five spots from where CSS had them include Plekanec, Pominville, Stoll, Vermette, Paul Martin out of a group of 33. Out of 77 guys taken more than five spots ahead of where CSS had them, the best five are probably Derek Roy, Tomas Kopecky, Mike Cammalleri, Nick Shultz and Craig Anderson. That’s a pretty good list but it’s out of a group 77(!). It falls off pretty hard after that.

    20. YKOil
      July 20, 2010 at

      My approach would be simple:

      Step 1 – Take the top 90 names from the major lists (Redline, CSB, McKeen’s) and rank them (like I do for the top-15 at draft time every year). This should generate a list of ~ 150 names.

      Step 2 – Run those 150 names through Gabe’s NHLE, stack that list with the Step 1 list and redo the rankings – adjust per smell test.

      Step 3 – Take the resulting list of 150 names and create bracket groups.

      Step 5 – NOW the scouts come into play. Have them apply their ‘scouting knowledge’ within those brackets with their main job being that of a) obvious eliminations and b) creating their own lists

      Step 6 – aggregate scouts lists and see what falls out**, do a LOT of video scouting

      Step 7 – adjust for Bob’s list come draft time

      Step 8 – adjust strategy for those GM’s who have tells

      Step 9 – adjust list for flier picks (Zetterberg types)

      Step 10 – Draft

      ** it would be VERY important to get scouts who understand and know that any credit due is due all and no one person is responsible for bad stuff – if I have scouts trying to ‘game’ the system I would have to replace them

      I figure those 150 names would last the whole draft.

      I’m not sure that the system would produce drafts BETTER than the better drafts but I am certain (in my own mind anyways) that the system would produce far fewer BAD drafts overall.

      Btw – great stories Vic, also – per your post at LT’s – I am thinking that Dryden’s anecdote per scoring opportunities was simply a written record of what the brainier hockey minds were probably discussing back in the day. I have the Jacques Plante book and I am thinking there is no way a guy that brainy (Plante), who studied his position like a science (as he did), wouldn’t have ever talked about scoring opportunities vs. shots on net when talking shop with other guys who were into that sort of thing.

      Just my opinion anyways.

    21. DD
      July 20, 2010 at

      The reason why DiPietro wasn’t listed on the Hockey News CSB lists is because he made a late decision to opt into the draft after Central Scouting had already published their final rankings.

      After DiPietro made his official decision, Central Scouting decided to make an exception and added him to their list in a special update. That’s why he doesn’t appear in all the published rankings.

    22. Matt D
      July 20, 2010 at

      This talk about scouting and the effect of interviews has reminded me of a similar debate in my work. I’m an academic, and some of the top departments in my field (Princeton, for example) simply do not interview when hiring. At all. They believe (with quite a bit of empirical evidence as justification) that interviews generate far too much noise relative to signal. That is to say, interviewing tends to make your hiring decisions worse than they otherwise would be, since they end up making you base your decisions on completely irrelevant factors (like ‘great tits’, though in my field that would likely count against you, on the false assumption that attractive women can’t possibly be smart). These departments read candidates’ CVs and research, and base decisions strictly on the information they contain.

      I’m not sure this is the right approach, but there’s something to it. And the kind of interviews that are typical in my field involve a full day of talking to other academics, including a lecture, in order to assess whether the candidate is going to be good at a job that involves lots of talking to other academics and giving lectures. So if there’s too much noise relative to signal there, then there’s no way that a 15 minute conversation with a teenager is a good way to assess whether he will be good at playing hockey, a task that is nothing at all like talking to scouts and GMs.

      All that to say: scouts and GMs should check the stats, watch the tapes, and leave psychological testing to the professionals.

    23. Hawerchuk
      July 20, 2010 at

      I know a guy who sells personality testing to pro sports teams. (It’s not psychological testing, btw – HIPAA makes it very difficult to release a person’s medical information to a 3rd party. But personality testing is seen as phrenology and hence isn’t subject to any restrictions.)

      Anyways, he has a ton of clients, but he has never been able to convince that it’s anything other than a load of crap. And he makes bold statements like “So-and-so will fit into this team’s system because of his psychological makeup.” I suppose it’s possible, but Rasheed Wallace has played over 1000 games in his career, including three times in the deciding game of the NBA playoffs.

    24. July 20, 2010 at

      Psychological test could be a moderately useful tool if used correctly.

      I’d do this:

      1. Test IQ. IQ is positively correlated with success in life. Part of this is because higher IQ is correlated with better impulse control, i.e. a greater ability to choose long term gain over immediate pleasure. In short, higher IQ is correlated with better work ethic and a higher likelihood to train, etc. It’s not perfect, but other things being equal, a player with higher Iq is slightly more likely to succeed. That is, I’d only use IQ as one factor to differentiate between otherwise equal players. That is, if two guys were the same size, with roughly equal point totals, etc. (Really low IQ may be very problematic.)

      2. Do some testing for personality disorders and other psychological problems. (Also get a family history. That’s predictive, too: If dad has an alcohol problem, etc.)

      The idea behind this kind of testing should not be that you are rating candidates as being psychologically strong and weak, as having good attitudes and work ethic. Rather, you use this kind of testing to look to see if a particular player has something mildly or severely wrong with them -more common than you might think. It sounds cruel, but people with mild personality disorders, family problems, etc., are less likely to succeed in life.

      Again, this kind of testing is only mildly useful in differentiating player’s who are otherwise roughly equal according to more important objective factors: production in junior at a young age, skating ability, size, etc.

      But this kind of testing is more reliable than going with the guy you like. IMO, anyway.

    25. Deano
      July 20, 2010 at

      I have done some hiring and used a technique called Behavior Descriptive Interviewing (BDI). The job is analyzed for characteristics that are necessary for success. Questions are then written to have the candidate explain their actual experience wielding the desired skill.

      It has remarkable ability to separate wheat from chaff. In one case, what was amazing (to me) was that two incredibly dissimilar candidates excelled in the process. They each aced different necessary skills.

      The skills necessary for successful performance as a pro-hockey player are easier to find on video instead of asking, ‘Describe a time, late in a game when you personally prevented your opponent from tying the game.’

      The things that you can find in the interview are mostly irrelevant in this career as indicated by Hawerchuk’s Wallace comment and the fact that Sean Avery is employed at a fair value.

      Also, I think Hawerchuk’s/DesJardin’s NHLE’s have limited value as players deviate from the norm physically. (i.e. they will not reliably predict boxcars for Jordan Weal or Zdeno Chara.)

      …and Ryan O’Marra looks like a hockey player (as does JFJ).

    26. Deano
      July 20, 2010 at

      kris – I am not sure your criteria are relevant as the job candidates’ physical skills are so exceptional that the other things don’t matter.

      Anecdotally, there are tons of substance abusers, terrible individuals and men with rotten childhoods that manage to make it at the highest levels of pro sports.

      I saw Brendl play in junior and if he tested ‘unmotivated’ I would have still drafted him. For all of Schremp’s rumored attitude issues, he seems to have willingly swallowed everything that was put on his plate. We can question the value of POS’s last contract, but like Avery, he will likely land on his feet.

      Pro sports are such an exception that most methods don’t matter.

    27. Deano
      July 20, 2010 at

      sorry, ‘most methods don’t apply’ not ‘don’t matter’.

    28. July 20, 2010 at

      Agreed Deano.

      I said these tests are moderately useful to distinguish between otherwise roughly equal players.

      Some people with low IQ or personality disorders succeed, but it’s less likely that they will do so.

      It’s better than going with the player who comes across well in an interview.

    29. mclea
      July 20, 2010 at

      Pro sports are such an exception that most methods don’t matter.

      Everyone knows that there is a floor of physical ability that has to be met before you would even begin to think about these psychological factors. Either you have the physical make up to play sports professionally or you don’t. It’s really that simple. We’re talking about the top 0.01% of a profession. Take the top 0.01% of any profession, and the only thing they likely have in common is that they’re really good at doing whatever is they do. Everything else becomes a distant secondary concern.

      But everyone is trying to gain an edge, so you end up justifying investigating things that probably don’t matter. It’s like that everywhere, not just professional hockey.

    30. mclea
      July 20, 2010 at

      Some people with low IQ or personality disorders succeed, but it’s less likely that they will do so.

      I’m interested in hearing why you think IQ would be strongly correlated with success in professional sports.

    31. July 20, 2010 at

      I didn’t say “strongly correlated” just more strongly correlated than “seemed cool when he interviewed him.”

      IQ might not be predictive of success in sports. Who knows? But it’s predictive of success in general, it’s predictive of success in each individual effort, including hockey, unless and until we have reason to rule out hockey as being different from other endeavors. Moreover, I pointed out IQ is correlated with impulse control and the ability to delay gratification, which are obviously things that help athletes train to become better.

      I really should have links for all this stuff. Sorry.

    32. Quain
      July 20, 2010 at

      Now, even if the scouting department can cover the investment that the team makes in it – say it’s generating $2MM in excess value annually on expenditures of $1.5MM – teams should still be looking for better returns on their investment where it’s possible. Say, for example, the Oilers could pay speeds $100K annually and generate $1MM in surplus value. They’re up $900K instead of $500K and have an additional $1.4MM available to invest in other areas.

      I think this is a spot on description of what a team that can’t afford to spend the cap should be looking at. Frankly, if Nashville disbanded their scouting department and just drafted off the TSN list, would they be significantly worse off? Maybe. If they used the savings to spend another $1-2MM on a forward, would they still be worse off? I highly doubt it.

      However, if you’re Toronto, is this really the ideal you’d ascribe to? You can spend $60MM a year (discounting long-term deal shenanigans) on players, but if you’re still pulling in $50MM a year in profit (I’m making up numbers) and your owner wants to win, the calculus slides in favor of the scouting department — speeds is more efficient, but with money to burn you start caring more about absolute value when there exists very little marginal cost to the rest of your organization.

      I’m not sure how many teams would fall into the Toronto category, but there are certainly some who can max player salaries and have a bit of walking around money. If they hired speeds to do their drafting and could funnel the $1.4MM savings to the rest of their organization, would it be better spent? Does additional coaching, training, or draft pick development produce a better benefit per dollar? I’m not sure, but it’d be an interesting bunch of numbers if we could figure it out.

    33. July 20, 2010 at

      I have so much to say on this issue that I figure I may crash Tyler’s site. From proper use of resources, proper evaluation methods both scouting and statistically, CSB as a whole, outdated scouting methods and much more.

      That piece by Elliot topped by Gare’s book are just bare illustrations of how dire the situation is because it extends much further than that. As a hockey society, we’re preaching the wrong methods on how to evaluate a player. Case and point for Kirill Kabanov.

    34. Vic Ferrari
      July 20, 2010 at

      YK,

      Re Dryden, in the last couple of hours I’ve come to realize that you are almost certainly correct.

      This paragraph from 1972summitseries.com remarkably insightful, though. And would appear to be a mile ahead of the curve:

      In Ken Dryden’s/Mark Mulvoy’s 1973 book entitled Faceoff At The Summit, an interesting statistic was published: Shots at goal. Although no definition was given, it is safe to assume that shots at goal included actual shots on goal, as well as shots that were blocked, hit the post or cross bar, or went high and wide. Essentially it is a form of measuring scoring opportunities.

      That’s true, though only for even strength hockey.

      A decade later Ken Dryden completely and utterly contradicts this line of thinking in an interview. He talks about the Russian rope-a-dope system of 1972. Seriously, you can’t make this shit up. And that’s the delusional, out-to-lunch Dryden that I remember from reading “The Game”.

      He surely published that original remark in 1973, but as you say, it’s almost certain that he didn’t understand it.

      Some bugger(s) did. Who I wonder? They must have been compelling enough, or famous enough, that Dryden believed them in the moment, even though it flowed against his nature and his understanding of he game. Tretiak maybe? I dunno.

    35. July 20, 2010 at

      To come back to my point about Lowetide then, I’m not convinced that the Oilers scouting has found them a ton of players who have been on real value contracts in the past few years. Part of that is the team pissing away value years and maybe part of it is that the drafting has generally sucked but that 05-06 team had a lot of guys on value deals, many of whom were found by the scouts. Today’s team doesn’t.

      How much of that is scouting, how much of that is pissed-away years, and how much of that is the change in rules with the new CBA? All those value deals from the Cup run were signed under the old CBA, with a crappier dollar, and more owner-friendly FA rules. Unless you get something huge in the ELC (or submarine a guy’s numbers until his second contract, like they seem to be doing with Gagner), that ain’t happening at this point.

    36. July 21, 2010 at

      On Tyler’s point about margins, I think he’s bang on in most cases – any team unable to spend to the cap really ought to look at their scouting department as an area worth stripping down.

      On the other hand, as was mentioned above, if you’re the G.M. of the Maple Leafs, marginal dollars don’t really matter to you except insofar as they impact your salary cap.

      That said, it still seems likely that the money could be better spent elsewhere (on the AHL team, perhaps).

      Buffalo was way ahead of the curve when they moved to heavy reliance on video; travel costs more than salaries are the real burden for any scouting department, and by slashing that area they must be saving a ton of money.

    37. July 22, 2010 at

      @Jon

      I’d argue that the amateur scouting department is an area that teams should be spending more resources into as that’s the most critical starting point of consistent organizational success. Unless as Tyler says your scouts aren’t doing as good a job as Central then it’s a moot point.

      Also when drafting most people’s top-end of their boards tend to be relatively the same, but a way to perhaps better isolate their true ability is looking at the 2nd round and beyond.

      and @Tyler

      On Jeff Skinner, CSS was the only major service that had him that low. It was a wide consensus that he was a top 10-15 pick based on ability.

      A place I’d look to decrease spending may be in the minor affiliates and player development as it seems less and less significant prospects are spending large amounts of time in the AHL/ECHL.

    38. Sidney
      July 24, 2010 at

      Oilers should trade Sam Gagne for Matin Hanzal. Straight up. Hanzal a big center great at faceoffs very good 2 way player

    Leave a Reply

    Your email address will not be published. Required fields are marked *