Unfortunate Misconceptions: Judging the 2008-2010 Drafts
As some of you may remember, this past year I started a massive draft project that spanned the Polian era, from 1998 to 2010. However, I stopped after 2007, because I realized that the 1-3 years of data available (AV was only charted through 2010 at that point) didn't give me enough to evaluate the more recent drafts.
ProFootballFocus apparently has no qualms about the quick evaluations however, and has recently started a series that grades the 2008 to 2010 drafts for each team, starting with our beloved Indianapolis Colts.
While I appreciate the attempt to evaluate drafts, the method chosen by PFF has several big issues with it, leading to vast misconceptions.
- The grading is purely subjective. The players are placed in one of nine groups, ranging from +2.0 to -2.0 (0.5 increments). There is no base value or criteria by which the players are graded by, leading to inconsistent results.
- There is (largely) no consideration for draft position. The players taken in late Round 3 who get the same production as a player in Round 6 shouldn't get the same grade. Either somebody has great production, or someone has poor production.
- There is no context for the draft picks. How did other teams draft in a similar draft position? How was the overall talent level at this point in the draft? PFF pretty much throws context out of the window for the entire article.
But, all the complaining in the world is lame if you don't have a contrarian opinion to offer as a viable replacement. So, with the 2011 data now available for AV, here is my explanation of where PFF missed in their rankings.
PFF didn't have anyone in the top two groups for the Colts, the +2.0 (You've just found Brady in the 6th round) or +1.5 (Getting much more than you bargained for) group.
+1.0: The scouts nailed it!
Pierre Garcon, WR (205th overall pick in 2008): The scouts didn't just nail it with Garcon, but he was much higher value than they expected. Garcon is a definite candidate for the next tier up. While no Tom Brady, Garcon provided excellent value (6 AV/season) at a draft slot that doesn't often produce it (Average: 1 AV/season, Median: 0). A find like that in the seventh round is more than the scouts just being on top of things.
Pat McAfee, P (222nd overall pick in 2009): This is the right place for McAfee. While he provided tremendous value for the seventh round, punter just isn't as impactful of a position as others.
+0.5: Never hurts to find a solid contributor
Philip Wheeler, LB (93rd overall pick in 2008): Had Wheeler developed quicker, I would have clamored for his ascension on this list, but with his struggles in his first two and a half seasons, I can't argue with this placement. (His AV/season of 3 was slightly above the average of 2.1 for his slot)
Jacob Tamme, TE (127th overall pick in 2008): A slight err in consistency here for PFF. Tamme has produced slightly higher value than Wheeler (3.25 AV/season), but was picked 34 picks later. Tamme was instrumental in the 2010 playoff run, catching 67 passes and four TD's. He should be higher than someone like Wheeler, but PFF's system leaves him in no man's land between the 0.5 and 1.0 grades.
Pat Angerer, LB (63rd overall pick in 2010): While Khaled Elsayed is correct that Angerer's tackle count isn't indicative of his performance, Angerer's value is definitely worth a higher ranking. Just two years into the league, Angerer quickly earned the MLB spot, allowing the Colts to cut long time fan favorite Gary Brackett. Angerer has shown improvement throughout his short career, and more than doubled his draft slot's average value (2.8 vs. Angerer's 6 AV/season).
Ricardo Mathews, DT (238th overall pick in 2010): Mathews deserves to be here, as a seventh round pick just getting into games is a positive sign.
Kavell Conner, LB (240th overall pick in 2010): This is one place where the inconsistencies of the grading truly show. Conner was picked two picks after Mathews in 2010, but has started 24 games since then, and shown to be a starter worthy LB as a run stopper. He is not good in coverage, but has been adequate as a starting linebacker. This value is incredible for his late draft spot (Average of 1, Median of 0.5; Conner's AV/season is 5 so far), and for his grade to be the same as a player like Mathews is laughable.
Austin Collie, WR (127th overall pick in 2009): Yes, he hasn't looked the same since the concussions, but he's also had the Quarterback Trio from Hell throwing him the ball. Collie has had an average AV well above his draft slot's production (6 AV/season compared to 2.1 average and 1.0 median), and will likely be much more productive with Andrew Luck throwing him the ball. Collie's absolutely fantastic play in 2009 and 2010 alone show that the scout's nailed it.
0.0: It could have been a lot worse
Jerraud Powers, CB (92nd overall pick in 2009): Yes, Powers has struggled with injury problems, but he's been a much better player than Philip Wheeler, who was drafted in the same spot. Powers has been the best cornerback for the Colts since his drafting, and was easily the best pick of the next 15 picks in 2009. Powers' AV for 2011 is strangely missing, but he averaged 4.5 in his first two years, over 1.5 more than the average for the 92nd overall pick. He definitely deserves a positive ranking.
-0.5: That pick was not put to good use
Mike Pollak, G (59th overall pick in 2008): I completely agree with this grade. Pollak was disappointing, but not the huge bust he's often claimed to be. He has been a decent depth linemen, nothing more. His AV is slightly above average, but it's inflated due to his starting in 2008 and 2010, but being pretty bad he really didn't deserve to.
Marcus Howard, DE (161st overall pick in 2008): Was Howard disappointing? Yes. But his disappointment isn't due to his lack of production (he was about average for his draft slot), but because of Carl Nicks being taken just four picks later.
Tom Santi, TE (196th overall pick in 2008): Santi showed promise, which is good for a late 6th rounder, but his promise was derailed by injuries. I would have put Santi in the "It could have been worse" category, considering that he doubled the average output of his draft slot.
Steve Justice, G (201st overall pick in 2008): Justice was decent depth in 2008, but never fully recovered from back surgery over the offseason, and didn't play another down. He still outproduced his draft slot (although slightly), is the injury failure of a late 6th rounder the fault of the Colts?
Mike Hart, RB (202nd overall pick in 2008): His body couldn't take the NFL, but Hart ran fairly well when he got a chance in 2010 (4.3 YPC), and was a primary contributor to at least two wins that season (Kansas City in Week 10 and Houston in Week 13). For a late 6th rounder, Hart deserves to be up a notch.
Jamey Richard, C/G (236th overall pick in 2008): Richard was a bad starter, but for him to make the team and be a contributor for four years is fantastic for a seventh round pick. He vastly outperformed his draft slot (nearly four times the average AV), and also deserves to be higher up.
Donald Brown, RB (27th overall pick in 2009): Brown is improving, but still is slightly underachieving for his draft spot. Good grade by PFF.
Curtis Painter, QB (201st overall pick in 2009): "Given a chance to step in for the injured Manning, Painter looked every bit the type of guy you’d rather was holding a clipboard than throwing passes in the NFL." True, but he's a late sixth round for him to provide some value is a good thing. But, in the end, you can't really argue with this.
Jaimie Thomas, ST (236th overall pick in 2009): Thomas was average for his draft slot, which is little to no value. If you want to give him a slight negative grade for that, so be it.
Kevin Thomas, CB (94th overall pick in 2010): Thomas got somewhat a raw deal with a freak knee injury his rookie year that left him out all season, and then a shortened offseason in 2011. It's still to early to tell with him.
Brody Eldridge, TE (162nd pick in 2010): Eldridge isn't good. Neither are most late 5th round picks. Eldridge slightly outperformed his spot (1.5 to 1), and the next player off the board in the 2010 draft with real value wasn't until pick 210.
Ray Fisher, CB (246th overall pick in 2010): A no value pick. For pick number 246, that happens more often than not.
-1.0: What a waste!
Jacques McClendon, G (129th pick in 2010): McClendon was a project that failed. There were two possible picks that would have been better in the next 32: Kendrick Lewis and Perrish Cox. This was a wasted pick, but it's not like they missed out on something crucial. I'd move it up one level.
Terrance Taylor, DT (136th overall pick in 2009): Taylor was disappointing to be sure, but late 4th rounders usually don't produce more than marginal role players anyway. The only two notable players in the next 40 picks? WR Johnny Knox and DB Chris Clemons. Again, the Colts weren't missing out on anything here. Move it up.
-1.5: The scouts failed, big time!
Fili Moala, DT (56th overall pick in 2009): Moala was a bad pick, no doubt. The Colts could have had Terrance Knighton or Sebastian Vollmer. But, the value of late second round picks is highly overrated. Even with Moala's failings, he still has provided enough production to have more than a point higher AV average than his draft slots historic average. I believe that it's a very inflated AV, but even if you lower it by a whole point, he still is about average production. This isn't a huge loss, although it is a waste. I'd put him at -1.0 in the flawed PFF grading system.
Jerry Hughes, DE (31st overall pick in 2010): Terrible pick. While the logic behind the pick is reasonable, Hughes has been a relative bust. As I've said before, Hughes has been very poorly utilized by the previous coaching staff, but that doesn't change the fact that Hughes hasn't done well. His AV is one, while his draft spot average is nearly four and a half. He belongs in this group.
While the PFF analysis tends to agree with the majority of draft analysis, that doesn't make it good analysis. Failing to address the relative draft position, surrounding talent, etc. makes for lazy writing and poor insight. Draft analysis as a whole tends to do this, looking at the sum of the players and expecting to see stars. Well, it's awfully hard to draft stars when you're drafting low. Overall, the Colts numbers were above average in 2008 and 2009, and right at average for 2010. While they didn't draft extremely high totals, like 2006, their value for their relative position was fairly good.
Unfortunately, most draft analysts put all teams on the same scale, when their positioning just doesn't account for it. I expect a much better draft for the Colts in 2012, not because they have better talent evaluators (although that may or may not be true), but because they have much higher draft selections.
Great deconstruction of that PFF piece, Kyle. I feel like so many of these analyses are out to "punish" the Colts for not making more of the Peyton Manning era. Everybody knows that Peyton is a better QB than his brother and for Eli to have more rings feels wrong to people. They want somebody to blame and the latest fashion is to blame Polian's recent drafts. But writing an emotional opinion piece and calling it analysis illuminates nothing.
Keep up the good work. CA is fighting the good fight to separate fact from myth.
I'm curious how the Colts will compare with the rest of the league once PFF releases its analysis of the other teams' drafts. Seems like they graded pretty tough. I'd argue for quite a few of those rankings being a notch higher. But I think the basic flaw with the Colts' drafts in recent years has been the failure to find any really top notch players. They've gotten some good but not great players from the late round picks, but Angerer is the only above average value in the top two rounds. Nothing really special anywhere.
Draft position explains much of the drop-off from the days when Polian grabbed stars like Manning, Freeney, Wayne & Clark in the 1st round, but it has also been awhile since he found a Bethea, Mathis or June in the later rounds, or a UFA like Saturday or Brackett.
@ECB looking at the Rams today, it looked inconsistant. For example, he graded marquis Johnson and Fend I Onobun as 0.0, because they were just late round picks that didn't work out. Those same type of players on the Colts (Richard, Hart, etc) got grades of -0.5.
ESPN auto-draft did pretty good for me last year, i think we should give it a shot.
I agree with the rankings Pro Football Focus has given to those picks, and it's not the first time an analysis of the Colts' collapse in 2011 pointed to the poor drafting the management did from 2008 forward. Please don't use the excuse of draft position. Other top flight NFL teams (Patriots, Steelers, Packers, etc.) haven't had as many misses as the Colts had in the past four drafts.
@jtgolfer You agree with the rankings? So you agree that Ricardo Mathews, Philip Wheeler, and Jacob Tamme were as good of draft picks as Pat Angerer, Kavell Conner, and Austin Collie?
You're wrong about the other top flight teams. They have had better position in the drafts than the Colts, and it's really not even close.
@jtgolfer i think i can give you the steelers, but not the patriots. they've had many many MANY more misses. but they also had many many more picks, too, so could afford to miss more.
Question: If injuries forced a late round pick into a game, but did terrible, what does that do to their AV? I know late rounders are generally meant to create depth anyway, but if just playing in a game gets you AV, even if you're terrible, I would think that's kind of a faulty statistic, right? Is there such a thing as negative AV? Just curious to know more about it.
I would also like to 2nd DougEngland's thoughts on Jerry Hughes. I hope he can get motivated this year and be used properly by the coaching staff. If so, I can see him becoming a pretty good role player, hopefully similar to a lite version of Aldon Smith in SF.
@TheGreatMisdirect Every statistic has its flaws, but AV (as a whole) is pretty good. There are places where it may be off, but as a whole its pretty sound. Basically it judges the value you get from a player, not necessarily the talent of the player. So, yes, if a late pick gets in a game, and gets some measurable value, it will (likely) raise their AV.
But, think of it this way: if that player wasn't available, then what? Somebody has to make the tackles, catch the passes, and block. Even if a player isn't very good, they can still provide value. Obviously, a player that is terrible will have a lower AV than a good player, so there is relative accuracy there. That's one of the things that people forget about Tony Ugoh. Was he a poor pick? Yes, especially with the trade up. But, if he wasn't drafted, then who would have played left tackle? Ugoh provided very good value for two years, and you have to take that into consideration when evaluating the pick.
@omahacolt @Kyle Rodriguez @TheGreatMisdirect
You mean besides starting for a defending Super Bowl champion and playing pretty damn well as a rookie? Honestly I never understood what happened with Ugoh, but the latter years of his colts career were so bad that fans tend to forget that he was lauded as our LT of the future after his first 2 years in the league. He ended up being a bust but saying he was horrible for 5 years and provided no value at all is just indefensibly wrong.
Hughes seemed like an interesting prospect, but he just got pushed around and used against the run and did nothing against the pass in the few opportunities he's had. I would never use him in the RZ.
I stubbornly refuse to give up on Moala and refuse to call him a bad pick. (Damn what the stats say.)
And Jerry Hughes... I am still very interested to see what becomes of him.
