Pre-employment tests are used for a lot of different jobs—everything from entry level clerks to executives. One of the best known users of tests is a consortium of 32 companies that uses tests for their 1760 “front of the house” employees. I am speaking of the National Football League (NFL).
NFL teams have a very in depth selection process. They hire scouts (think of them as recruiters) who are in charge of finding the best college football players in the country (high school players cannot go directly to the NFL, per the collective bargaining agreement between the players and the owners). Sometimes the players are easy to find as they play for teams in conferences that show all of their games somewhere on TV. Or, they travel to some remote outposts to see players who are on teams with less notoriety.
There is one thing you should know about the NFL owners. For being a small group of very rich men (with a couple of exceptions) who made their fortunes in the free market system, they’ve adopted a very socialistic model for the league. This model includes a provision where they choose talent from the labor poor in reverse order of how good the team is (called the draft). So, instead of competing for the best talent, the team that did the worst the previous year gets to choose first, then the next worst team, all the way to the winner of the previous season’s Super Bowl (championship game). This means that mistakes are very costly as 31 more players get picked before a team can make another choice.
The league also puts together days where all of the teams can watch the players run, throw, jump, etc. College teams have “pro days” where their players train in front of the NFL scouts (all are invited).
As you can see, the NFL teams have the opportunity to observe the players perform the job they are likely to be hired for over at least a 3 year period (maybe longer in some cases). They also get to put them through laboratory situations to measure specific abilities. And, since the 1970s, the league has administered the Wonderlic Test to the players as well.
The Wonderlic is a pretty straight forward measure of intelligence. Not surprisingly, players from colleges (e.g., Ivy League) that are pickier about admissions do better than those who will take pretty much all comers. Also, offensive players tend to score higher than defensive ones. The league doesn’t have any hard and fast rules on using the results, so teams could choose to use it (nor not) as they please. Personally, my big concerns about the testing program are:
1) There’s not any data supporting that it’s predictive (see Lyons, et al, 2009)
2) I realize that pro football players are in the public view, so they don’t have the same level of privacy that the rest of us do. But, do they really need their scores on an intelligence test made public?
3) If we assume that the cognitive abilities required of a quarterback are different than a middle linebacker, wouldn’t we need separate validation studies/cut-off scores for each?
News now comes that the NFL has paid for the development of a new test (read about it here). I find it amusing the NY Times shows it with a graphic which refers to a Rorschach test, which is nothing like an intelligence test. But, that’s just lazy editing.
The NFL is generally keeping the test under wraps. It was funny how the agents for the players complained that their clients couldn’t practice the test. Imagine a friend of a job applicant being upset because his/her friend couldn’t see your pre-employment selection instruments before hand. It is good to know that the developer got some input from subject matter experts. There are a lot of test security issues, so I can how validating the test on current players would be tough. A study with college players would also compromise the test in that they will be the ones taking it soon.
There’s a lot of temptation for using any pre-employment test in this kind of recruiting situation. There are only so many good players that can help each team, and you may not get the chance to draft the ones that you really want. Any edge in evaluating players under these rules is a huge competitive advantage. I think the only question is whether there is something on a test that tells the team something unavailable through all of the game film. The answer is probably yes–The ability to perform at a higher level when there is more pressure and the game is played by faster and more talented people. Each subsequent performance level comes with a new set of challenges, and professional football is no exception. If a team can use any test to figure that out better than the other teams, they will be more successful in selecting players.
Predicting human performance is an inexact science. But, as the NFL shows, there are significant competitive advantages to doing it better than your competitors. What are you doing to ensure you have this advantage?
For more information on pre-employment testing, test validation, skills assessment, and talent management, please contact Warren at 310 670-4175 or warren@allaboutperformance.biz.