Are The Masses Valid?

Creating content specific knowledge tests is a time intensive proposition.  When the company using them does not anticipate a lot of openings, the return-on-investment of those tests can be questionable.  For these reasons, I am taking a foray into crowd-sourced tests.  I’ll be using the, hopefully, for selecting software developers.  I’m not being coy about the website, but I’d like to have some data and experience with the process before naming the company.

How do these tests work?  Multiple-choice items are written and submitted to the platform.  Then, people (applicants, incumbents, and the curious) take them online.  Each item is evaluated based on its difficulty based on the crowd taking it.  Using advanced analytics and the items’ difficulty, the platform’s algorithm gives the person items until it can accurately estimate his/her skill level.  It is all very similar to computer adaptive testing models.

What’s great about it?

  • It is free (for now) to both the user and the company.  This makes for a high ROI, assuming it’s valid (more on that in a minute).
  • There is a high level of sophistication in the scoring.  This reduces the number of items a person has to take.
  • Since everyone takes a different set of test items, it maintains a high level of test security.  A friend may pass on to me certain questions, but there is no guarantee that I’ll see those items.
  • Results can be fed directly into an applicant tracking
  • A wide base of people can write items, so it is likely to cover more content then if written by one set of job experts.

What makes me hesitant?

  • I am concerned about the quality of the test items.  Maybe some items appear difficult because there is potentially more than one right answer.  Of course, this could be fixed via user comments.
  • While the items appear to measure job content (I’m no expert for a lot of them), it doesn’t mean that they measure those things that really distinguish between high and low performance.  Of course, this is a problem for all content-validated tests, but it’s of particular concern here as each test taker will be getting different items.  For this reason, I’m not going to use the platform to select candidates until we collect enough test data to do a criterion-related validation study (statistically show that scores on the test are correlated with job performance) and set test norms.

I am excited about this new adventure in testing.  There are computer-adaptive approaches to some tests, primarily in the educational realm, but this is the first that I know of for such specific tests.  If successful, it will streamline the implementation of skills/knowledge tests for several of my clients.

What are your thoughts on using crowd-sourced tests?

For more information on legal pre-employment testing systems, skills assessment and talent management, please contact Warren at 310 670-4175 or [email protected]

Are the New Ways of Fishing for Candidates Valid?

As the US economy continues its slow recovery, I am sure that some of you are in recruiting mode.  I don’t claim a lot of expertise in this area, except to say that I’m always concerned when recruiters “feel” they have a good candidate without any validated methodology to back it up.

This article delves into the changes that have occurred in online recruiting, particularly for small business, though many of the concepts would apply for large ones as well.  Online recruiting was one of the first great services that the internet provided businesses.  The job/recruitment sites that sprouted online immediately allowed many more companies to search many more candidates (and vice versa) fairly quickly and easily.  That business has now matured and companies have made great strides in creating their own recruiting areas on their webpages.  Candidates and companies can also take matters more into their own hands via LinkedIn and other social media sites.

An interesting conversation is taking place among employee selection specialists about how much, if at all, employers should evaluate a candidate’s personal social media (Facebook, Twitter, Instagram, etc.) as part of the recruitment/selection process.  I do not think there is that much valid data there, with the exception of how well a person maintains their network.  However, there is potentially a missed opportunity in a person’s more business-oriented social media presence.

In specialized fields, it is likely that your candidates are “connected” with others in your company in that field.  While many of these connections are casual, some are not.  Where one of your employees is linked with a candidate, you should ask the employee about the person. 

This is a good time to note that you will want to have a policy statement to your employees about how business-oriented social media will be used in the recruiting process.  For instance, letting employees know that while you won’t be searching their profiles, their names may appear while evaluating candidates.  Allow them to opt-in to be contacted if their name appears as a connection to someone you are recruiting.

These profiles have a lot of verifiable information (school, degrees, awards, previous employers).  More importantly, the profiles can indicate a person’s passion for the field, willingness to reach out to others for knowledge, and depth of interest in a profession.  These traits, and others, may be valuable.  Of course, if you are going to screen candidates on their business-oriented social media, you will want to validate the information by seeing if the data correlates with success with your current employees.  This will help you make better decisions and be able to defend the practice, if challenged. 

Do people make themselves look their best on social media?  Of course they do, but probably no more than they do in resumes or interviews.  Even with that in mind, there is a great opportunity to evaluate candidates on job related criteria online.

For more information on legal pre-employment testing systems, skills assessment, and talent management, please contact Warren at 310 670-4175 or [email protected]


Knowledge Be Damned

There was quite a bit of hoopla a few weeks ago when Google announced that they were no longer going to ask wacky questions in their interviews or use GPA as part of their hiring practices for most jobs.  This interview with their head HR executive provides a bit more detail about their thinking.

My reaction to this was, “No kidding.”  The research data on the effectiveness of using behavioral interviews is plentiful and I don’t know why they just didn’t use Google Scholar to look it up.  For a company built on knowledge, it makes not sense to me why they don’t use well established research to guide their HR practices.  Even if they thought their culture was special and previous studies did not apply, I’m surprised that it took them this long to figure it out given how many people they have hired (the same can be said for looking at GPA or their brain teasers).

This is not so much about Google as it is about applying established knowledge in organizations.  Every company thinks that there is something special about how they do business.  And they are probably right.  Where they miss the mark is when they think there is something unique about their culture that suggests that established hiring practices will not work for them.  Guess what?  The (imprecise, but effective) science of predicting job performance has been built over time and across a myriad of organizations.  Tools should be refined, but wheels do not need to be reinvented.  This means that the most effective techniques/tools, such as cognitive ability tests, job simulations, behavioral interviews, and valid personality tests are going to be more effective than unstructured interviews, education, GPA, and previous experience for every job.  How you apply them should be based on a job analysis.

Innovation and creativity is critical in selection and HR.  We always want to be in tune with our clients’ needs and changing technology.  However, these things do not make valid practices obsolete.  Rather, it makes them important building blocks.

For more information on legal pre-employment testing systems, skills assessment, and talent management, please contact Warren at 310 670-4175 or [email protected]

Thanks for coming by!

Please provide this information so we can stay in touch.