For North American sports fans, there is not a more public selection process than that National Football League (NFL) preparing for the annual talent draft. This is the process for allocating new players (rookies) who have finished their college careers to the teams. Players cannot sign a contract with any team they choose until after they complete their rookie contract with the team that drafts them. Players not chosen in the draft can sign with any team.
Besides evaluating players based on their college games, the NFL teams also invite the top players to be evaluated at what they call a combine. At the combine, players get interviewed by teams and are put through a variety of physical and medical tests. Teams use all of this information to compare players against each other (by position) so they can make the best choices during the draft.
Of course, in reality, the top draft choices are made mostly based on the players performance in college. Players at the best schools compete with and against other players who are likely to be drafted, so watching them perform in a game tells teams pretty much what they need to know. And, as I wrote about last week, there is a big bias towards players who went to the “best” schools. But, the teams do use information at the combine to inform them about players who they don’t feel they have good data on. For instance, those who are recovering from injuries or played at schools that don’t compete against the top schools.
There’s only one problem: There is very little data that supports that the “tests” given at the combine of predictive of success in the NFL. This article about the problems in measuring hand size in quarterbacks provides just one example of that.
One can see how this all got started. Quarterbacks need to be able to throw a ball well (with a lot of speed and accuracy) and to be able to hold on to it under pressure and having a large hand (as measured from the tip of the thumb to tip of the pinkie) would seemingly be related to both of those. But, it’s not. All quarterbacks grip the ball a little bit differently, regardless of hand size, to get the best results. The article suggests that hand strength is the better predictor of quarterback performance and that it is unrelated to size. But, those who evaluate quarterbacks just cannot let the size measurement go.
I am guessing that most of your organizations have an unproven selection myth, such as, “Our best managers have gotten their MBAs from one of 10 schools” or “Good supervisors are those who have worked their way up in our industry” or “Our most successful programmers had previous experience at specific companies.” I used to hear, “Our best call center agents have previous experience before coming here” all of the time. But, when I conducted validation studies in contact centers, it was rare that previous experience was a good predictor of future performance. These myths are easy to evaluate, but changing HR practices is harder. It often requires good data and a shift in culture to change thinking. However, moving on from myths is often required to make better talent decisions.