Monday, August 30, 2010

Selection vs. Development Assessments

This post is written for the everyday manager, HR manager, coach, or consultant that doesn’t have the time or interest to learn about validation, reliability, confirmatory factor analysis, correlation coefficients, and adverse impact. However, you’re using assessments, and you know just enough to be dangerous. Oh yeah, that pretty much describes me. Maybe you too.

I’ve written about assessments for leadership development before. While there are lots of them to choose from, the common ones used for development tend to be 360 degree assessments (multi rater) and personality preferences (DISC,  Kolbe, Hogan, FIRO-B).

Most of my work involves development assessments. That is, the manager takes an assessment, the results go back to the manager, and a coach helps to interpret the results and come up with a development plan.

Here is where I see managers and organizations get into potential trouble: they want to take that favorite development assessment and use it to make selection decisions. They may get a copy of the assessment from a naive or unethical HR manager or coach, or, they might just “ask” the individual directly for a copy.

In most cases, it’s done with good intentions. Managers want to make a smart hiring or promotion decision, so they are trying to learn as much as they can about the candidate. Also, if they do end up hiring the person, they can get a jump-start on their development.

I’m guessing some of you are already doing this, even if you’ve been warned not to.

So what’s the harm?

Here’s the problem:

I’ll start with the legal/HR stuff you’ve probably already heard: if it’s not tested for “validity” (it measures what it’s supposed to measure) and “reliability” (it’s consistent over time), you could get your #%* taken to court and sued.

A disgruntled candidate could claim that you used the results of that assessment to make your hiring decision and it was bogus. Therefore, selection assessments require a stack of research and are held to a much higher standard than your average free online personality assessment or horoscope. And let’s face it – if someone takes one of these things for development – and it’s bogus – who cares? However, if someone doesn’t get hired or promoted because or it, it’s a big deal.

It’s been my experience that “we could get sued” doesn’t always stop a lot of maverick managers from taking risks. Well, good for you. So let me give you a more compelling reason: if you use an assessment that has not been “benchmarked” to the position you’re hiring for, you could end up making a stupid hiring decision.

That is, you really don’t know what the candidate’s assessment profile is telling you. You may end up favoring somebody because of personal preferences (or bias) - that have nothing to do with performing well in the job.

The concept of benchmarking is really pretty simple. It works better when you have a lot of incumbents already in the position you are hiring for. You just need to administer the assessment to the top 10 performers and the bottom 10 performers (without telling them that they are the 10 worst of course). Then, look for differences between the best and worst performers, and establish an ideal profile for the position. That way, you’ll identify the stuff that really matters for success in that specific job.

Make sure you have benchmark for each job, even if you’re using the same assessment. A success profile for an engineer looks very different than one for a sales role.

If a candidate falls above or below the ideal profile, it does not mean they couldn’t do the job – it just means it could be harder for them. If anything, it gives you an idea of what you need to poke at during the interview process.

Now, you could use the results of a selection assessment for development if you end up hiring the candidate. However, usually selection reports don’t include that kind of detailed information –you’ll need to purchase a development report for an additional cost.

BTW, for those of you that that are the extreme other end of the continuum - that is, that organizations should NEVER use personality assessments for selection - you're wrong. Most organizations use them, and as long as they are benchmarked (validated), it's perfectly OK. Except for the MBTI - which measures psychological type..., not personality. Like I said, just enough to be dangerous. (-:

Finally, a manager should never use the results of any one assessment – even one that’s properly benchmarked – to make a hiring decision. That’s just plain lazy and ineffective. Assessments should also be combined with good old-fashioned selection interviewing and reference checking.

I hope this helps de-mystify the difference between development and selection assessments. Yes, assessments are can be good tools for development and selection - just don’t get the two of them mixed up, or you won’t be doing yourself or your company any favors.


davidburkus said...

Great post. I too lament how many assessment are not tested for validity or don't correlate to anything...yet companies shell out tons of money to use them.

Brent said...

I don't get the difference between a psychological assessment (MBTI) and a personality assessment ( DISC, etc). Can anyone tell me what the key difference, applications are?



Dan McCarthy said...

David -
They sure do. Hope this helps.

Brent -
MBTI measures psychological type - which I am told is different than personality, and should never be used for selection. Why? Damn good question.... it's one of the reasons I don't even like using the MBTI, becuase it's too complicated with too much potential for misuse. It also requires certification to administer.

John Hamilton said...

Great article - the difference between assessment and development centres is something I spend much of my time explaining to organisations. Now I can point them here!

The key to employing any personality measure is firstly to make sure that the tool is valid and reliable as you say but also make sure that the person who is interpreting the profiles really knows the instrument and is suitably qualified AND experienced in doing so.

Crucially, although many organisations often do this, decisions can never be made in isolation i.e. based on the personality profile alone. Without a sound feedback interview and using the profile in conjunction with other data and information, perhaps from a competency interview or other assessment techniques, you cannot be in a position to make a decision about an individual's suitability for a role.

Remember, most personality questionnaires are about "preference" or "style" not about "ability". Just because I prefer to focus on the big picture rather than attend to the minutiae doesn't mean that I can't do the detail - so, although you might be right not to hire me for that admin job, it would be wrong to do so based on the profile alone.

As for MBTI, it's a great tool for coaching and team building - very powerful to use when there is conflict in an organisation or one needs to understand the dynamics of a team, for example. But it's more of a clinical exercise and, to answer Brent's question, critically, there is no way of linking the MBTI's dimensions to job behaviour or performance which is why OPQ, WAVE, Hogan and several other Occupational Assessment tools are far more appropriate for selection.