Grantland's Bill Barnwell has a great piece about his time at the MIT Sloan Sports Analytics conference where he surprisingly found Eric Mangini a kindred spirit when it comes to sports analytics. Mangini noted that there is a value to intangible qualities players sometimes possess, they are often overstated and should not be relied upon when making critical decisions. Case in point:
Of course, the other thing worth considering is that a scout might be able to better measure a player's on-field performance than his psychological makeup. Mangini recalled that one old scouting report suggested that a certain defensive back suffered from a lack of toughness that would retard his progress at the professional level. That player? Four-time All-Pro Rodney Harrison. Oops.
That's a nice story and it sort of proves the point. To be more direct, what Barnwell and Mangini are saying is that the answers teams (and fans) need regarding prospects are on tape and in the interviews.
A few examples clearly illustrate the problem with relying upon intangibles when making personnel decisions. The conjecture about Tim Tebow's near-mythical leadership qualities are meaningless: it's about how he plays the game. Marvin Harrison was too small, slow and too quiet to be a team player. Peyton Manning "choked." Rodney Harrison lacked toughness. Unfounded rumors of drug abuse caused the top 11 teams in the 1995 draft to question Warren Sapp's personality flaws. Ryan Leaf was viewed by some as a "winner." Indeed, those judging intangibles in lieu of what is on the tape will often find themselves making poor decisions.