In validating tests, getting a hold of good individual performance data is key. But, it is also one of the more difficult parts of the process to get right.
Intuitively, we all think we can judge performance well (sort of like we all think we are good interviewers). But, we also know that supervisor ratings of performance can be, well, unreliable. This is so much the case that there is a whole scientific literature about performance appraisals, even as there is currently a movement within the business community to get rid of them.
But, what about objectively measuring performance (for every new account opened you get $X)? If the Wells Fargo imbroglio tells us anything, it’s that hard measures of performance that are incented can run amok. Also, while they are objective, single objective measures (sales, piece work manufacturing, etc.) rarely reflect the entirety of performance. Lastly, for jobs where people work interdependently it can be very difficult to determine exactly who did what well, even if you wanted to.
So, what’s one to do?
- Establish multiple measures of performance. For instance, call centers can measure productivity (average call time) and quality (number of people who have to call back a second time). Don’t rely on just one number.
- Even when a final product is the result of a group effort, each individual is still responsible for some pieces of it. If you focus on key parts of the process, you can find those touch points which are indicative of individual performance. Again, look for quality (was there any rework done?) and productivity (were deadlines met?) measures.
- Objective performance measures do not have to have the same frequency as piece work or rely on one “ta-da” measure at the end. Think of meeting deadlines, whether additional resources were required to complete the work, etc.
- Don’t get bogged down in whether or not a small percentage of people can game the system with objective measures. We seem OK with rampant errors in supervisory judgment, but then get all excited because 1 out of 100 people can make his productivity seem higher than it is. If you dig into the data you are likely to be able to spot when this happens.
When I hear people say that you cannot measure individual performance well, I cringe. Of course you can. You just need to know where to look and focus on what is important.