Performance of Support Functions

There’s an interesting confluence of factors occurring in the National Football League (NFL) right now.  The NFL has locked out its referees over a pay and benefits dispute.  Why they did so is a whole other blog post.  The league is now employing referees from the lower college and semi-professional ranks (replacement refs).  What I find interesting about it is the effect on the performance the NFL and its players.

Using typical performance measures (scoring, number of penalties, number of overturned calls from instant replay, etc.) the replacement refs have not affected the game too much.  Anecdotally (as a football watcher and reading commentaries from others), they seemed to do pretty well in week 1 but not so well in week 2 (though this may have been biased by a particularly difficult Monday night game which was broadcasted nationally).  Since attendance and television ratings have not gone down, one could say that the performance of the league has not been affected.

As the owners are locking out the referees, they’ve been pretty quiet on the subject.  The players are not impressed by the replacement refs and feel that they don’t do enough to protect player safety (never mind that it’s the players who make the work environment unsafe for each other).  The players are not so concerned that they have talked about staging a sympathy strike.  They have, and will continue to, adjust and scoring hasn’t changed.  So, their performance has been unaffected.

The league office wants to have it both ways.  For years they talked about the integrity of the game and how the referees are part of it.  Now they are saying that they can hire and train anyone in the field to do the job of a select few that they used to say were the best of the best.  All of this will change if a player gets seriously injured on a play that is somehow linked to the performance of a replacement ref.

We can look at the referees as a support function in a more typical organization.  For instance, no one thinks too much about IT until something goes wrong.  And management thinks their current support functions are doing great until they find a cheaper way to have those services delivered.  What the NFL is saying is that their key support function can falter and the product is relatively unaffected in the eyes of the customer.  They are gambling that they can fix the problem before the business (TV ratings and attendance) is affected.  I’m not sure that the rest of us would want to function under this business model.

The lesson for HR is that external customers are far less concerned about support processes than we are—which is a good thing.  However, even when our management creates a difficult situation, it’s HR that often has to create the people fixes (creating bench strength, making quick hires, implementing training, etc.) so that a break down in process does not eventually lead to a breakdown that affects external customers.  We can hope for the best and do these things on the fly, like the NFL.  Or, we can plan and have processes in place to ensure that the performance of employees and our organizations are maintained when a difficult situation occurs.

For more information on pre-employment testing, skills assessment, and talent management, please contact Warren at 310 670-4175 or

Caveat Emptor

Let’s say you’re about to purchase an employment related test.  It could be for hiring entry level personnel, help make a promotional decision or for providing feedback to people on their skills.  There are about 50,000 of these available and several that purport to measure the same things (leadership, EQ, etc.), so how do you choose the right one?

Professional standards require that test publishers provide important information about tests so that buyers can make informed choices.  Of course, not everyone selling tests, especially on the internet, bothers with such niceties.  At a minimum, here’s what you should find out about a test before you buy it (note that some of these may require you consult with someone with good knowledge of statistics):

  1. What is the validity of the test?  Validity means the relationship between scores on the test and doing the job.  For practical purposes, there are two ways to show the validity of a test:a) Is there a statistical relationship between test scores and job performance?  Remember correlations from college statistics?  What you are looking for here is a statistically significant (large) correlation between how someone does on the test and how well they do on the job.  Look for correlations of about .30 and higher.  This provides evidence that the test does what it’s supposed to do:  measure how well someone can do a job.   This type of data is critical when you are looking at personality or aptitude tests.

    b) Do the items on the test represent the job?  If you were looking at a test for an electrician, you would want to see items about Ohm’s law, different phased motors, etc.  Or, if you were looking for a manager assessment, you’d want it to involve coaching, providing directions, etc.  This is more than just the test looking right.  They key here is that the content of the test should match the content of the job you want to use it for.  Test publishers should be able to show that the test items were written based on input from job subject matter experts.

  2. What is the distribution of scores on the test (or the average score and standard deviation) for people in the same job as those you are going to give the test to?  This is how you can tell if a person’s score is good, bad, or average.  Saying that someone scored a 37 on a test is meaningless without this information.
  3. Is there a difference in scores between different demographic groups on the test?  Test publishers should be able to let you know if group (gender, race, age) differences exist on the test.  This doesn’t make a test good or bad, but it lets you understand whether particular passing points might lead to adverse impact.
  4. You’ll want to get some administrative details as well.  What’s the time limit? Does it have online and paper-and-pencil versions?  If online, how is feedback given?

If the test publisher doesn’t have this information available, it means they have not done the proper R&D work on their test and you shouldn’t buy it (unless you want to be the R&D site).  Remember, once you administer a test to a candidate or employee, it’s your responsibility and you need to live with the outcomes.  Be sure you have full knowledge of the tests that you are using.

For more information on pre-employment testing, test validation, skills assessment, and talent management, please contact Warren at 310 670-4175 or

No, This Will be Good For You

Is receiving feedback rewarding?  Perhaps, if it’s positive.  But, feedback providers tend to borrow from behaviorist psychology (Skinner, etc) in assuming that getting the feedback will result in learning, much in the same way that rewarding a behavior will.

In most cases, performance or 360 feedback is not nearly immediate enough to be linked to specific behaviors.  Also, the feedback from these processes tends to focus on what a person should be doing (which can be vague) as opposed to rewarding good behaviors (which leads to learning).

So, how can the feedback process lead to more learning?  Here are two tips:

  1. Focus more on rewarding good behaviors than pointing out poor ones.  Not only will this help the person receiving feedback, but if others see which behaviors are rewarded they will want to emulate them.
  2. If you point out a behavior that should be changed, point out what the better replacement behavior is and then reward the new behavior.

Some people prefer the “brutally honest” approach to feedback.  I’m sure you’ve heard something to the effect, “We’re all big boys and girls here, tell it like it is.”  There are a couple of problems with this approach.  One is that there is research on 360 feedback (Smither, J. & Walker, A.G., 2004) that those who received a relatively small number of negative comments (in relation to positive ones) improve more than those who receive a relatively large number of negative comments.  You could argue that strong performers got that way because they are better at receiving negative feedback and improving, hence fewer negative feedback comments.  However, another interpretation is that people who get LOTS of negative feedback lose interest in improving.  Perhaps they decide that the system is rigged against them or that there is no way they can get better at everything or they just don’t believe that they are that poor at their jobs.  Either way, being “brutally honest” in spades comes with costs.

Another reason to ease back on the negative feedback is the pain it causes.  I’m not exaggerating.  Neuroscience studies (Eisenberger, N., Lieberman, M. & Williams, K., 2003) show that the brain reacts to negative feedback the same way it does to physical injuries.  I think it’s fair to say that no one comes to work for that kind of experience.

Remember, insight alone won’t change behavior (otherwise, therapists would be out of business).  If your feedback is solely focused on pointing out what’s wrong with a person and you don’t reward them for doing what is right, the people receiving the feedback will always treat it like a trip to the dentist rather than a learning experience.

For more information on employee engagement, skills assessment, 360 feedback, and talent management, please contact Warren at 310 670-4175 or

Does Employee Development Work?

You might think that’s an odd question given how much time and money companies spend on it. However, they rarely go back and examine whether or not performance improved or what they can do to ensure that it does next time.  Why is that?

Part of the reason is that organizational development specialists aren’t really trained do so.  Also, since the need for training is intuitive, management is not hard pressed to justify the value of it.  Note that this becomes a real problem when budgets are cut as training is almost always one of the first things to go.  Why?  Because its value hasn’t been documented.  Acquiring these kinds of measurement skills or resources would actually help organizational development professionals keep their jobs as they would then be spending time/money on initiatives that work and could show their value.

I had an employee development project with a client where we had the data to measure the results. First, we measured performance and assessed key skills. Then the group received training based on overall results and individuals received training based on their individual needs. After eight months time we went back and measured performance (see below).

What did we find? There were some areas of performance where there was significant group improvement. In other areas there were not (there wasn’t a decrease in performance in any area). However, we now know which areas are still trouble spots for the team and which ones are not. Therefore, training dollars and time can be spent in areas where they are needed.

I’ll be clear about my bias here:  Money spent on valid pre-employment or promotional assessments bring a much greater return than money spent on development.  However, there will always be need to improve the skills of your current workforce.

Training/Development initiatives should be approached with the goal of improving performance (and not doing better on a test).  Once the performance area is identified, you should be able benchmark current levels and compare them to post-training levels to provide an indication of success.  Sure, a control group would also be nice, but not always practical.

Evaluating the effectiveness of employee development isn’t easy, but it can and should be done. To learn more about how to do it, or this particular project, send me an e-mail using this link.

For more information on  skills assessment, and talent management, please contact Warren at 310 670-4175 or

Thanks for coming by!

Please provide this information so we can stay in touch.