When Should You Stop Using an Assessment?

I’m guessing that you would not expect me to write about discontinuing the use of a pre-employment or pre-promotional assessment.  But, there are instances when it is appropriate to do so.

For instance, the National Football League (NFL) has decided to stop using an intelligence test that they had been using for years to evaluate new players.  I have written about the league’s use of the test before, so I won’t rehash the arguments about it here.  However, its reasoning for not using it any more really comes down to:

  1. They did not feel it was predictive.
  2. It led to a poor candidate experience (which, to the NFL means bad publicity).
  3. And those are two very good reasons not to use a test.

Another reason to discontinue the use of a test is when knowledge, skills, abilities, or personal characteristics (KSAPs) required of a job change.  At some point, administrative assistants stopped typing pages of documents, so a test of how quickly someone could manipulate a keyboard no longer made sense.  Changes in customer dynamics can impact KSAPs as well.  When working with a call center client, our validation data showed that personality tests that predicted performance for those taking phone calls were not effective for those who took customer inquiries via e-mail or chat.  This led to a change to how the tests were scored depending on the open position.

This does not mean you should automatically drop using assessments because a job changes or has converted to WFH from an office position.  However, knowing that for many people WFH is the new normal, it may be time to see if the work has really changed and the if that impacts the KSAPs.  If the status quo has held, you have your answer.  If there are some changes, then another validation study is likely in order.

The use of assessments, like many HR procedures, tends to take on a life of its own.  Once they are in place, there is a lot inertia (we have always done it this way) keeping them there.  It does not have to be that way.  A good job analysis and validation study can help you modify your testing tools so that you get high value from them.

Do Women Have Less Managerial “Potential” Than Men?

Whether it is for succession planning or leadership development, many organizations are concerned about a person’s potential.  That it is, in the future, will be this person be performing at their current level (at least relative to peers) or is there something about them that will lead to an acceleration in their performance so that they’ll rise above many others.  This is often reflected in faster promotions as well.

The idea that managers can use their judgment to distinguish between who is a good performer now and who has potential is ridiculous.  What objective information would tell you that an average performer now will blossom in 5 years?  Initiative? Curiosity?  Of course, these things impact current performance, so they really are not related to potential if we assume that the two are not the same thing.

The other example frequently given is whether a person in a given role can perform the next role up.  For instance, can someone in a technical job supervise others or can a current manager be an effective executive.  This has less to do with potential than whether a person has a skill set that they have not been given an opportunity to show in their current role.  Note that there are plenty of valid ways (Assessment Centers being one) to determine whether people have supervisory/management/executive skills which are far more accurate than judgment.

Organizations find potential is sexier than evaluating skills, so potential gets a lot more attention.  And, since objectively measuring potential is time consuming/expensive and no one holds executives accountable for being wrong about potential, we get people sitting around and making up reasons why employee A is a “high po” and employee B is not.  You will not be shocked to find out that this approach is loaded with bias against women.

Whether it is due to unconscious bias or stereotypes that go against women being as self-promoting as men, female employees (at least according to the large study cited above) get the short end of the potential stick.  For example, in that research, women had higher performance ratings, but lower potential ratings.  This leads to fewer relative promotions for woman compared to men.

Now, it could be that the lower potential ratings were justified.  Perhaps men with similar potential ratings were performing better than the women down the road.  In the study the opposite was true.  Women with lower potential ratings outperformed men in the future.

I think we can draw a few conclusions from this study (and common sense):

  1. Organizations that are using potential ratings should look at their impact on women (and minorities).  They may not like what they see.
  2. No matter how much rating systems are dressed up (I’m looking at you, Nine Box), they are subject to huge amounts of biases and stereotyping.  Organizations that use this approach should look at the accuracy of the judgments and only use those who frequently are right.
  3. If you are interested in potential, define it (e.g., promotions, pay increases, etc.) and look at what predicts it over the long term after controlling for sex (since those promotional decisions are likely biased as well).  Then you have a system for understanding potential.

Organizations should plan for succession so that those being considered get the training and development opportunities they need to succeed.  However, using human judgment to identify who has potential and who does not is a fool’s errand.  It is unfair to women, which means it is wrong and negatively impacts your business.  There are ways to predict future performance in your organization.  You just need to put in the work to discover them.

Many thanks to Dennis Adsit for sending me the cited article.

Bring me Your Best Empathetic Narcissist!

The 24 hour news cycle (including sports) and social media has people caring a lot more about the comings and goings of CEOs than is probably necessary.  This is particularly true when they jump/get pushed from their perches.  When this happens, it is up to the organization to find that JUST RIGHT person to lead them going forward.  Suddenly, selection matters to them (momentarily).

CEOs (and head coaches of sports teams) have unusual jobs and career paths.  Typically, they rise in their profession based on technical skills and as they move along. Then, these skills are supplanted by their ability to influence others inside and outside of the organization.  A coach who develops players becomes a fundraiser.  An engineer who runs a tech company has to schmooze investors. A banker who leads a financial institution needs to explain the company’s investments in not-so-popular industries.

All the while, those who aspire to be CEO are competing with others for the opportunity.  Sometimes these competitions are very public, but other times it takes years of skill development and checking the right boxes (and acquiring allies) to put oneself in position to be considered.  And don’t forget building the ego to think that you are the “one.”

So, what makes for a good CEO?  According to this article, you need to be a smart, strategic, narcissistic, and empathetic.  Oh, and don’t let people see the narcissistic part.  But, let’s not assume that there is a one-size fits all model here.  Among other variables, if the company is successful, the CEO should fit the culture.  If the organization is struggling (or is facing significant threats on the horizon), someone who can shift the culture may be more effective.

Boards of directors will do better in their search when they can be clear about their expectations for the CEO.  However, they also need to be aware of the quirks that it takes to be successful in the job. 

Thanks, as always, to my colleague Dennis Adist (@DennisAdsit) for making me aware of the article.

Taking Mythology Out of Resume Screening

Ah, so much for the summer blogging break.  I have so much to say about WFH and the delta variant whiplash.  But, that is for another time.  What I have been thinking about lately is how there is so much available talent (people looking for work and to change jobs) and what employers are doing about it.

Typically, this kind of environment is great news for companies that are hiring.  That is not entirely the case now because, since there is so much employee movement, job candidates seem to have the upper hand in terms of salary and WFH flexibility.  However, employers do have a lot of options as well.  The only problem is that some are over playing their hand.

Case in point is the use of resume screening software.  Don’t get me wrong—this type of tech is something that companies need to use.  It is an efficient and objective way to go through resumes.  However, as this article (thanks to Denis Adsit for sending this my way and the link requires a subscription) points out, employers are likely missing out on lots of good candidates.  It is not because the algorithms don’t work.  It’s because they filled with untested assumptions that are provided by hiring companies.

It is amazing how many myths companies have about who they hire.  For instance, each time I have done a validation study in a contact center, line managers insist that previous experience is a plus.  And each time the data does not support that assumption.1  If you attempted to validate similar assumptions, I am sure that you would find that fewer than 50% were actually good predictors of performance.

When these myths are plugged into resume screen algorithms, they help to screen out people randomly.  This means that you have fewer resumes to read, but it also means that the ones you are reading are not better or worse than the ones you don’t.

Another problem with the data companies give to the algorithms is that that the choices are draconian because they are used as a thumbs-up or thumbs-down screen.  A better approach would be one where certain elements are given points (again, based on a validation study) and a cut-score is used.  For instance, let’s say that your algorithm selects out people who have had more than 3 jobs in 5 years.  You might be missing out on people who have several other very attractive things in their resumes.  And you would potentially be interviewing people with fewer attractive things on their resume but stayed at a job for 5 years.  Is that one thing really such a deal breaker at this stage of the process?

The other hurdles that companies place in front of candidates in the algorithms are unnecessary educational requirements.  I’ve written about this before, so I won’t get into again here.  However, if you are going to validate other assumptions about what is predictive of success on resumes, you should do the same regarding educational requirements.  This will widen your pool and likely lead to an equally, if not more, qualified pool and one that is more diverse.

Resume screening software is a very useful tool in pre-screening resumes.  Like any other computer program, they are only as good as the data that goes into them.  By ensuring that what you provide the algorithms is based on fact rather than myth, you will get a lot more out of the screens.

1 If anything, my experience shows that for contact centers, the opposite is true—those who worked in them before do worse in the next job.  Why?  Well, if they were good at the job, they would not have made a job change in the first place.  Also, there is a lot of unlearning that has to go on when training veterans on the customer management software.

Does Lowering the Bar Always Mean Lowering Standards?

Recently, I’ve wrote about how IBM was looking to remove college degree requirements for many of their jobs.  The rationale behind this change was to remove employment hurdles that may be leading to racial discrimination while not being indicative of future job performance.

Since that post, we’ve seen how employees/job seekers are now in the driver’s seat in terms of employment choices and salaries in ways we have not seen in many years.  There are a lot of factors that are causing this, including:

  • People wanting to keep the autonomy WFH gave them.
  • An economy that was pre-COVID and is creating jobs faster than the population of the US is growing.
  • Social pressure on some companies to provide a “living wage.”
  • Generous COVID unemployment benefits, relative to pay, in low wage states.

At some point, companies are going to be challenged as to how they can fill some of their jobs.  Paying more almost always helps, but unless you are paying the most for people of a certain education/experience level, you’re eventually going to lose that battle.

When you are experiencing labor shortages for your positions, it’s generally best from a talent perspective to make you pool wider rather than diving deeper into it.  One way to do that is to recruit from spaces, schools, etc. where you haven’t before.  Another way to is to re-examine your degree and experience requirements.

Yes, that’s right, removing unnecessary degree requirements can improve your talent pool and reduce discrimination.  Here’s my favorite quote from the article:

“By creating your own dumb barriers, you’re actually making your job in the search for talent harder.”

Yes, this does put some more responsibilities on Learning and Development departments as well as hiring managers.  Some may see this as a return of more investment in employees.  Others may see it as a gap filler between what new hires are taught in school and what they need to know.  Future data will tell whether this improves engagement as well.

Note that this approach should not affect whatever validated pre-employment assessment you are using.  The knowledge, skills, and abilities required are not going away—only the assumption that the only way to obtain them is through a degree program.  There are instances where necessary skills are more readily obtained in school (yes, I’d prefer that my surgeon has gone to a medical college and served a residency).  But, there are many jobs, especially in computer technology, where this just is not always the case.

At some point, the labor market will swing back the other way and employers will have a bit more leverage.  However, we are at a unique moment where companies can alter their paradigms.  They have the opportunity to re-evaluate some requirements that will give them, and job seekers, more choices while reducing discrimination.  Or, they can try to dig their way out of a hole while they are still standing in it.

Post-COVID WFH Conflict: Bosses vs. Just About Everyone Else

As the medical need for remote work is coming to an end, the conflict surrounding it may just be beginning.  This difference in opinion between executives and workers who can do their work just as well from home as in an office is just beginning to surface.

For instance, this report (note that it was conducted pre-vaccine) shows that on almost every key metric regarding WFH, executives are much more eager to have people back in the office than the employees. I do think that some of this difference is based on full-time school being available for children, so this gap may narrow come fall. Interestingly, executives cite that they feel that people need to be in the office a certain amount of time to maintain a distinct company culture and despite them saying that performance had improved since COVID.

When it comes to employers wanting workers to come back to work, there is really a sense of trying to put the toothpaste back into the tube.  In Los Angeles, for example, last week it was reported that 24.4% of officer workers were reporting to their work location on a given day. The rate in April of 2020 was 21.6%.  That’s not a big move, especially considering the high vaccination rates of college graduates in LA County. 

While there is some (not a lot, mind you) data that suggests that people work more hours and are somewhat less efficient at home, there are a lot of people who do not want to go back to commuting or spend extra time getting ready for work.  They would rather see their kids when they come home from school and having more control over their day-to-day schedule.  Companies that keep a higher level of flexibility are going to have a huge recruiting advantage, especially for experienced talent. Lower tenured employees tend to want to be in the office more than longer tenured ones—either due to not having kids yet and/or feeling a greater need to schmooze more to keep their careers moving.  This tells us that when flexibility and autonomy are part of your culture, you’ll have a post-COVID edge on attracting talent.

It is this last point that I think executives are missing.  When they say maintaining “culture,” many of them are really saying, “Going back to the way things were that I am comfortable with.”  You can have a customer oriented culture via Zoom.  You can have a quality oriented culture via Teams.  What you cannot have is a, “I can only tell if you are working if I can see you” culture via WebEx.  And, presuming that leaders feel there is a relationship between culture and success, they are going to have a difficult time arguing that WFH has impacted culture but not productivity.  What does that say about your culture when success survives without it?  Why would a talented person want to work for a company that has a culture of control for control’s sake?

Most executives understand that the hybrid model of flexible work schedule for white collar workers is here to stay.  The bigger question is going to be whether they accept it begrudgingly or accept it as an element of an evolving culture that emphasis employee flexibility and autonomy over leader control.

Removing Unnecessary Employment Barriers

Let’s play some DE&I trivia!  As many of you know, the landmark case in employment discrimination is Griggs v. Duke Power.  But, what was the aspect of Duke Power’s hiring that got them into court?

If you said their use of pre-employment tests, you’d only be partially right.  The decision was also based on the use of discriminatory educational requirements (in this instance, a high school diploma).  Interestingly, after that tests got a bad name, but companies continued to use school credentials with little or no problem.

As the US economy and culture pushed more and more students towards college, racial disparities in educational attainment have persisted.  Yet, companies rarely questioned whether asking for high school or college degrees for certain jobs really gets them better candidates.  In some cases, this requirement is a classic “like me” bias?

Of course, the only way to see if a high school or college degree is necessary for a job is to conduct a job analysis and compare that knowledge, skills, and abilities with a high school or college curriculum.  Yes, I want my surgeon to have an MD, thank you very much. Far too often companies have used degrees as a de facto job requirement without ever thinking about its impact on organizational performance (are we turning away qualified people?) or fairness.  This is particularly true in IT where there are many self-taught people in the field.

Due to a confluence of factors, some big companies have rethought their use of degrees as qualifications.  Besides this leading to potentially more diverse hiring, it will also save them money (but be an economic boom to the new hires).  Whether it would lead to less college enrollment and lower higher education costs is certainly possible.  More importantly, it would lead to a paradigm shift of associating all white collar jobs with college degrees.

One can argue that getting a college degree shows tenacity and commitment over a long period of time.  And I would agree.  But, there are other ways to show this as well.

Change only comes when we do things in a different way.  And solutions to long term problems often require big actions.  Removing high school or college degrees as job qualifications when they are unnecessary removes a significant barrier to employment for racial minorities that could have an impact at your company.

Assessing Talent is HARD—Ask the NFL

The National Football League (NFL) is holding its annual talent draft tonight (as I write this).  So, yes, you’re getting two sports related blog posts this week.  What’s fascinating about this is how the 32 teams evaluate a large pool of potential players and how wrong they are in the most critical evaluations.  If you’re familiar with the NFL draft, you can skip ahead past the bullets.  If not, here’s a quick primer:

  • Each year, the NFL forces the top players who have not yet played in the NFL (think of guys who played college football) to participate in a process where teams select which players that they want (the draft).  The order of the draft is determined by the teams’ records the year before, with the worst team (Jacksonville) going first and the team that won the championship (Tampa Bay) going last.  There are 7 rounds in the draft.
  • If a team wants a particular player, but thinks that he will be taken before it is their turn, they can trade players and/or their draft picks for a better pick.  This is called trading up and can be very expensive.
  • The most important employee (player) on an NFL team is the quarterback.  Teams that do not have an “elite” player at this position will do almost anything to draft one.  There are economic incentives for drafting a quarterback rather than trying to get an experienced one from another team.
  • Each team has an entire department of people whose sole job is to evaluate the skills and abilities of college players, determine the needs of the team, and recommend players to draft and in which order.  The NFL even sets up what are essentially job fairs where the teams can look at players up close, interview them, and they even give a league-wide cognitive ability test.  These players really get put under the microscope.

So, we have a situation where the skill and ability levels of the players are well known.  Given that, you would think that teams can select players, especially quarterbacks, with uncanny accuracy.  And you’d be wrong.

If we look at the quarterbacks drafted in the 1st round (63) since 2000, it’s pretty fair to say that (not including those who are too early in their career to judge):

  • 15 turned out to be very good (consistently led their team to a winning record and won playoff games)
  • 13 were/are good (there teams won more than they lost, but were nothing fabulous)
  • 30 were not good (losing record, didn’t stay with the team long, etc.). 

Those of you who are football fans may quibble with the categories where I put the players, but even if we move players around a bit between the categories, teams have about a 25% success rate and about a 50% miss rate.

In roughly the same time period, teams have “traded up” 25 times to draft a quarterback in the 1st round.  In other words, they paid a premium to “get their guy.”  14 of them were/are not very good, 3 were/are good, and 6 were/are very good.  Again, we see this 50% miss rate, though slightly higher hit rate.

How can this be?

Most obviously, football is a game where all of the players on the field need to do their job well for the team to be successful.  Like it or not, the quarterback gets outsized credit and blame for the team’s success and failures.  But, regardless of his skill level, he cannot control a lot of what goes on around him.  This makes for a very difficult selection situation.

There are also some confounding factors.  The first being that the worst teams pick before the good ones and they are more likely to choose a quarterback than good teams.  And, one of the reasons these teams are not very good is that they don’t judge talent well and do a poor job of developing and retaining the good players that they have.  Related to this, they often take a quarterback before they have good supporting players around him.  It’s entirely possible that several of the quarterbacks who were drafted early, but performed poorly, would have been successful with other teams.  And, a few of them, but not a lot, were after they changed teams.

To make the success rate seem even worse, teams choose to ignore players are other positions that are more likely (historically) to be successful in order to make the sexier, and riskier, play for a quarterback.  And I get that—there are enough examples of a highly drafted quarterback completely changing a team’s fortunes that teams are willing to miss 2 out of 3 times to get one.  Any team would be happy to draft 2 not so good quarterbacks to get to the one that leads them to a championship, even if it takes them several years to do it.

What we do know is that new quarterbacks tend to succeed where the team has a sustainable plan.  Often times, teams change their playing philosophy if they don’t have a successful year so they fail to implement one that fits the quarterback’s skills.  Successful quarterbacks rarely play for multiple head coaches (managers).  The lesson here is that good plan that senior management sticks with is more likely to be one where your most expensive talent can succeed.

It used to be that newly drafted quarterbacks would have a more experienced one ahead of him to learn from for a year or two before being elevated to leading the team.  There are many reasons why that is not the case anymore, but it does limit the opportunity for some drafted players to develop into good quarterbacks. Players drafted at other positions tend to have the luxury.  Organizations that realize which key players can contribute immediately and which ones need some seasoning are also more likely to get the most out of their talent.

The NFL (and other professional sports) draft is a unique selection system.  In some ways it allows talent evaluators to get a great deal of information, but in other ways there is a lot of mystery around what allows players to translate their college success into the pros.  And, in looking at success vs. failure of the quarterback, sports fans tend to ignore the organizational issues that allow some players to shine rather than flounder.  All good lessons to keep in mind as companies begin to hire recent college graduates.

And good luck to Trevor Lawrence.

Moving From Counting to Progress

As I settle in to watch the Los Angeles Lakers play tonight in the National Basketball Association (NBA), my biggest concern will be about LeBron James’ health.  I won’t give a second thought to who is officiating the game and as to whether one of the officials is a woman.  OK, for the sake of this blog post I did check, and one of the 5 female NBA referees is working the game.

The reason I bothered checking was this article about female officials in the NBA.  The league is seen as more progressive than its North American counterparts (football, baseball, hockey, and soccer) when it comes to employment and social justice issues.  I was curious what the league was doing to encourage more women to become NBA officials and how they supported them.

The first thing that struck me was that the league establishes multiple pipelines for attracting referees.  If you have been a ref in one of them, you can get a look from the NBA.  And the league is not concerned whether they had officiated men’s or women’s games.  As long as you put in the time and you were good, you have a chance.

Once in the NBA, both male and female officials work both the top level and development leagues (the latter known as the G League) in order to gain more experience.  A little less than half of the full time G League referees are women.

So, the NBA throws out a large recruiting net (colleges, international leagues, etc.) and provides ample opportunities for the new officials to hone their skills, even when working with the top players.  Of course, the publisher of the article is also a broadcaster of NBA and WNBA games, so I wasn’t expecting any hard hitting accusations.  But, the NBA’s reputation for inclusiveness seems to have paid off in terms of the smoothness of the transition to having more women officiate games played by men.  It also means that the league does not have to bring out the trumpets each time a woman is a game referee.

The lesson from the NBA is that proper planning and culture can lead to diversifying workforces.  Taking steps to ensure high quality development and performance can make the revolutionary turn into the ordinary.

Are We Entering the Age of the Employee?

As working age people have been getting their COVID vaccinations in the US, companies are moving from the theoretical regarding the “new” work life into putting new policies into place.  There are a few I want to point out because they may be indicators of companies moving towards policies that are messier, but more employee focused.

Regarding work from home schedules, or lack thereof, General Motors came out swinging with Work Appropriately.  As their CEO puts it, “This means that where the work permits, employees have the flexibility to work where they can have the greatest impact on achieving our goals.”  So, the policy is basically, “Be an adult.  If you would rather not commute and you get can your work done, do it at home.  If you are a social animal and feel you’re more productive at the office, we’ll see you in the morning.”  This policy places the responsibility, where it should be, on the employee to manage his/her/their performance and career as well as their work schedule.

Many companies struggle with people taking their paid time off (PTO).  Even during the pandemic when many were experiencing additional stress, PTO was not fully being used.  Sure, part of that was due to travel being restricted and there are cultural issues to be addressed if a large number of people are not using this benefit.  But, many people were working longer hours from home and taking less time off.  Organizations tend to believe that people are more productive and engaged when they take their PTO and are often frustrated when they do not.  And, typical of American culture, they responded by threatening punishments (you can only accumulate so much PTO, use it or lose it, etc.).  Now we are seeing the pendulum swing back as companies are beginning to offer incentives for taking PTO.  Full disclosure: my wife works for an organization which has always done this and it helps.  She would use less of her PTO without the incentive. And I think this is the case in organizations that have particularly competitive cultures.  Incenting people to take PTO will not by itself reduce burnout, but it can be helpful.

Lastly, I want to bring up Amazon’s declaration that “We are going to be Earth’s Best Employer and Earth’s Safest Place to Work.”  Of course, this comes with the caveats that it came from an outgoing CEO and right after a bruising union fight.  However, that this additional employee focus, and not just for white-collar workers, was put on the table represents a sea change for an organization that is (proudly) customer-centric.

Now, this may just be a moment.  Senior managers, who felt the stress and disruption of the pandemic as much as their employees, may be viewing their “most valuable asset” differently now, but when the usual business pressures inevitably return, they may snap back to the status quo.  Or, employees will use these new tools to be productive and engaged enough so that they will stick.  We will soon see if we are entering the age, or the fad, of the employee.

Thanks for coming by!

Please provide this information so we can stay in touch.

CLOSE