.png)
What is predictive validity?
Predictive validity is a measure of accuracy used in science and psychology. It’s essentially a means of judging how accurately something can predict future outcomes.
When it comes to hiring, predictive validity is used to see how effectively an assessment can predict someone’s job performance.
If you’re going to be spending on assessment software, you’ll want to make sure the assessments they use are actually predictive.
Using predictive assessments will enable you to reliably identify those most suitable for the job - and the sooner you use them in your hiring process, the more time you’ll save.
Which assessments are the most accurate?
Make no mistake - for the most part, any assessment software is a step in the right direction.
Why? Because traditional CV screening is prone to bias.
We make snap decisions about people based on their identity and background and once we make our minds up, we tend to seek information that confirms our assumptions.
This is why candidates from underrepresented backgrounds tend to be disproportionately overlooked.
So what does work?
Whilst predictive validity studies can take years to complete, we can look at the results of the Schmidt-Hunter meta-analysis for a rough guide to what works and what doesn’t.
This landmark study summarizes 100+ years of research around hiring assessments.
As you can see from the chart above, assessments that revolve around background aren’t going to tell us much about someone’s ability.
A step up from this is cognitive ability tests. These are undoubtedly a fairer, more accurate means of testing skills.
Where these kinds of tests fall flat, however, is their relevance to the job itself.
A cognitive test might be great at identifying people with certain skills or behaviours.
But there’s no concrete body of evidence that suggests this makes them the perfect person for the role - the results of the test are still just a proxy for on-the-job ability.
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
Work samples, on the other hand, are designed to simulate the job itself by having candidates perform small parts of it.
Whilst there are many ways to test someone’s skills, work samples (which are sometimes referred to as work simulations), test these skills within the context they’ll actually be used.
In the Schmidt-Hunter study, work samples are actually used to test the skills of manual workers. However, given their predictive power, we adapted them for office-based jobs…
The idea behind work samples is to have candidates essentially do the job before getting it.
Whilst we can’t have candidates physically join us for a day at the office to see how they handle tasks, we can ask them to do small parts of it.
At the screening stage, writing-based tasks are ideal. If the job itself doesn’t require writing skills, be sure to let candidates know they’re not being tested on their spelling and grammar.
When it comes to interviews, you can have candidates actually role-play tasks to simulate them more closely.
For commercial roles, you could have them give a presentation or simulate a client call.
And for more strategic roles, you can present candidates with a larger case study to work through.
How to create work samples
The most effective way to implement work samples is by tying them to the skills needed for the job.
Start by listing 6-8 essential skills/behaviours. These can be a mix of technical and softer skills.
Once you know the skills you’re looking for, you can think of scenarios and tasks that would test them. For example, you could give candidates a list of tasks to prioritize, an email to draft or a tricky situation with a customer to think through.
For the most predictive work samples, use the tasks that candidates would actually be doing should they get the job. Are there any that can be performed via writing? If not, simply ask candidates how they would deal with them or ask them to explain their approach.
Once you have your work samples, you’ll need to give yourself some basic criteria to score them against.
This will allow you to give answers number scores and build a candidate leaderboard.
Your scoring criteria doesn’t need to be very detailed, a few rough and ready bullet points will do - you just want to write down what a good, bad and mediocre answer might include.
Do you need assessment software to find the best talent?
The truth is - you don’t need to use assessment software to start testing candidates with more accuracy.
Work samples can be used completely independently of the Applied Platform.
If you’re not ready to ditch CVs entirely, we’d suggest using 3-5 work samples alongside your usually process. Ideally, you’d score answers before looking at CVs.
You’ll likely find that those who you shortlist with work samples wouldn’t have been chosen based on CVs alone.
So, why use assessment software like Applied at all?
Faster workflow
Whilst you can (and absolutely should) try using work samples on your own, using software like ours will make creating, distributing and scoring work samples easier.
For high volume roles, we also provide multiple choice tests to narrow the initial pool.
.gif)
Anonymization
We know that unconscious bias leads to minority background candidates being disadvantaged, so we anonymize applications.
Again, although this is possible manually, it can be incredibly time consuming.
Work sample library
One of the most labour intensive steps towards using predictive assessments is actually coming up with the questions.
Thats why we built our work sample question library to help you get started.
Debiased scoring
Once candidates have answered your questions, Applied will anonymize candidates and assign them to your hiring team to review - with the order randomized each time to avoid lesser-known ordering effects.
These scores will be used to generate your candidate leaderboard.
Analytics
This is the one thing that would be near-impossible to do manually…
Using assessment software will allow you to see which questions are the most predictive and allow you to see any diversity drop-offs in your process.
Applied is the essential platform for debiased hiring. Purpose-built to make hiring empirical and ethical, our platform uses anonymized applications and skill-based assessments to identify talent that would otherwise have been overlooked.
Push back against conventional hiring wisdom with a smarter solution: book in a demo
.png)
