.png)
What is a situational judgment test?
A situational judgment test poses hypothetical workplace scenarios, usually with either multiple choice answers or a scale of how likely a candidate is to take a given course of action
By understand how a candidate would choose to respond to a realistic situation, organizations can make more informed decisions about how suitable they’d be for the job.
Where a typical interview or screening question might ask how someone has dealt with something in the past, situational judgment questions get candidates to think as if they were already in the role by asking them what they would do.
Example situational judgment question
Q: A customer is complaining about an issue they’re having. Which action should be avoided:
- Spending too long listening to them
- Politely asking them to calm down
- Suggesting that they speak to a manager
- Promising to follow up on their complaint
Most situational judgment questions are highly job-specific and will lay out varying degrees of context.
Whilst some are fairly to-the-point like the example above, others will provide candidates with a more complex scenarios that might involve not only customers but also other colleagues.
Although the example above gives candidates the choice of four answers, some questions might ask how likely they are to consider each option os a scale of 1-5.
What do situational judgment questions get right?
A situational judgement test is unquestionably a fairer, more predictive means of talent assessment than traditional methods such as CVs, unstructured interviews and even psychometric tests.
Predictivity
Your typical hiring process is centred around experience. Candidates with the most extensive, impressive-sounding experience generally tend to come out on top.
Although this may not sound too controversial, both years of experience and education are actually fairly poor at predicting how well someone will perform on the job.
A situational judgement test doesn’t make the mistake of over-indexing on experience and instead poses questions hypothetically.
This means that someone with the right skills - but not necessarily experience - has the opportunity to showcase what they can do, without their background holding them back.
Whilst there is evidence that suggests past behaviour does tell us something about how someone will perform on the job, the problem is that it’s extremely difficult to gain any valuable insights about past behaviour just by asking candidates about it.
When it comes to ‘tell me time when’ questions, we all know exactly what is expected of us.
This reduces these kinds of questions to an exercise in repeating the skills listed in the job description.
Candidates who are either proficient at lying or at least stretching the truth are more likely to be rewarded.
And what about those who haven’t encountered particular decision before? Does this mean they wouldn’t be able to do the job?
By leaving assumptions at the door and directly testing candidates with the very citations they’s actually encounter on the job, you’re far more likely to identify genuine talent.
Fairness and diversity
Relying on candidates’ background to inform hiring decisions isn’t just inaccurate, but also unethical.
Russell Group university degrees and experience at big companies may be an indicator of hard work or raw intelligence…
But they’re also signifiers of socioeconomic background.
Rather than attempt to account for this - which would be a monumental challenge, if possible at all - a situational judgment test simply tests candidates on the skills needed for the job without the use of these flawed proxies.
Since situational judgment tests don’t have to be tied to a candidate’s identity, they give provide everyone with a fair and equal chance to showcase their skills.
We know from decades of research that when we remove opportunities for bias to influence decision-making, diversity tends to improve as a byproduct.
And the more emphasis you place on skills (as opposed to background), the less biased your process will be.
The idea isn’t that you intentionally hire people without experience.
More often than not, you’ll find people who have done the job before.
However, you’ll be hiring people based on skills gained through experience and not just for the experience itself.
What are the disadvantages?
One of the major pitfalls of the situational judgment test (at least as its commonly implemented) is the use of multiple choice questions.
Although this may make high-volume roles much easier to manage, it doesn’t lead to the most predictive scores.
Often, the right answer is fairly obvious and with many real-life scenarios, there will often be more than one way to deal with it effectively.
If you're dealing with extremely high volumes of candidates and feel you need to use multiple choice answers, questions that ask candidates to quantify how likely they are to follow through with each possibility are a better option.
The more room there is for choice, the easier it will be to build a leaderboard and make decisions. Using simple multiple choice answers, you might find that too many candidates have similar scores to shortlist one over another.
How to improve situational judgment tests
Here at Applied, we use an assessment method similar to situational judgment questions called ‘work samples’.
Originally used to hire manual workers, we adapted work samples for desk-based jobs.
Why? Because they’re the most effective form of assessment there is!
Similarly to a situational judgment test, work samples take realistic scenarios and pose them hypothetically.
Where these assessments change is predominantly to do with how they’re answered.
Unlike situational judgment questions, work samples use open-ended answers. For most tasks, there isn’t a right answer per se, we want to see how candidates think and work through tasks.
Work samples can also take the form of job simulations. These are less hypothetical, and more about having candidates actually perform a given task.
For a sales role, for example, you could have candidates give a presentation or role play a call with a prospect.
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
| Mission / Values Alignment | Culture Fit Alignment |
|---|---|
| Passion for your company’s missionA shared approach to working/collaborating | A shared educational, cultural or professional backgroundSimilar demographic (age, social class etc)Mutual hobbies/interests |
To generate objective scores, we score answers on a 1-5 star scale.
Here’s what our scoring criteria looks like:
Using work samples to both screen and interview candidates at Applied, we’ve found that +60% of people hired would’ve been missed via a traditional process.
Aren’t work samples too time-consuming for both candidates and hirers?
We would never deny that work sample questions will require more time to both complete and review, at least in terms of time spent upfront.
However, by using a more demanding assessment method, you’ll likely see a reduced volume of more skilled, dedicated candidates.
It’s also worth noting that 5 work samples (the most we’d recommend using) needn’t take more than 45-50mins to complete. Answers don’t need to be extremely long or well-written, often we’ll let candidates know that bullet points will do.
If you’re able to screen candidates more accurately, you’ll also find that time is saved later on in the process. The more predictive your shortlisting is, the fewer interviews you’ll need to conduct.
When it comes to roles with high volumes of applications, we’d recommend using multiple choice questions as an initial screening tool, then followed by your work samples.
How to create your own work samples
Work samples are most effective when made specific to the job you’re hiring for.
The more closely questions simulate the job, the more predictive they’ll be.
Choose 6-8 core skills needed for the job
The first thing you’ll need to do is decide which skills you’re looking for. The ideal outcome is a mix of 6-8 hard/soft skills. For a marketing role, for example you might have technical skills like SEO or PPC as well as general working characteristics like creativity and time management.
Think of scenarios/tasks that would test these skills
To create work sample questions, take a handful of skills and think of scenarios that would test them. For example, you can ask candidates what they’d do of a project was going to be delayed to test skills like prioritization and communication.
Give yourself criteria to score against
For each work sample, make note of what a good, mediocre and low effort answer might include. This can be as simple as a few bullet points for each end of the spectrum. For the most accurate scores, have three reviewers score independently.
Applied is the essential platform for debiased hiring. Purpose-built to make hiring empirical and ethical, our platform uses anonymized applications and skill-based assessments to identify talent that would otherwise have been overlooked.
Push back against conventional hiring wisdom with a smarter solution: book in a demo of the Applied recruitment platform today.
.png)
