Structured interviews and scorecards: comparing candidates fairly
An unstructured interview mostly measures whether the interviewer liked the conversation. Two managers meet the same candidate and come out with opposite opinions, and a month later nobody can say why one person was chosen over another. A structured interview with a scorecard turns that impression into evidence you can compare.
What a structured interview is
A structured interview has three fixed parts: the criteria you assess, the questions you ask to assess them, and the scale you rate the answers on. Every candidate for the same vacancy gets the same questions in the same order. Interviewers may ask follow-up questions to get to concrete facts, but they do not invent a new interview for each person.
Research in work psychology has found for decades that structured interviews predict later job performance better than free conversations. The reason is simple: when everyone answers the same questions and is rated against the same description, the differences you see are more likely to be differences between the candidates rather than between the conversations.
What goes on a scorecard
- Four to six criteria taken from the job itself, for example customer handling, reliability, working with numbers or leading a shift. More than six and interviewers stop distinguishing between them.
- One or two questions per criterion. Behavioural questions ask about the past ("Tell me about a time you had to calm down an upset customer"); situational questions ask about a realistic case ("A delivery is missing an hour before opening. What do you do?").
- A rating scale with anchors: a short description of what a 1, 2, 3 and 4 look like for this criterion. A four-point scale has no comfortable middle, which forces a decision.
- A weight per criterion if some matter more than others, agreed before the interviews start.
- A field for evidence: what the candidate actually said or did, in a sentence or two.
- A recommendation at the end, such as hire, maybe or no hire, filled in last.
How to run structured interviews step by step
- Write down what the person in this job must be able to do in the first three months, together with the hiring manager.
- Turn that into four to six criteria and agree the weight of each.
- Write one or two behavioural or situational questions per criterion and an anchor for every point of the scale.
- Decide who interviews, which criteria each interviewer covers and how long the interview takes.
- Brief the interviewers: same questions, same order, take notes of facts, no scoring during the conversation.
- Straight after each interview, every interviewer fills in the scorecard alone, with evidence for each score.
- Only when all scorecards are in, hold a short debrief: compare scores criterion by criterion and discuss the largest gaps.
- Decide, record the reason for a rejection, and keep the scorecards with the application.
Worked example (illustrative numbers)
A retail chain hires a shift supervisor and uses four criteria with weights: customer handling 30%, reliability 30%, leading a small team 25% and working with the till and cash 15%. Two interviewers rate each criterion from 1 to 4, and the scores of both are averaged.
- Candidate A: customer handling 4, reliability 3, leading a team 2, till and cash 3. Weighted score: 4 × 0.30 + 3 × 0.30 + 2 × 0.25 + 3 × 0.15 = 3.05.
- Candidate B: customer handling 3, reliability 4, leading a team 3, till and cash 3. Weighted score: 0.90 + 1.20 + 0.75 + 0.45 = 3.30.
- Candidate C: customer handling 4, reliability 2, leading a team 4, till and cash 2. Weighted score: 1.20 + 0.60 + 1.00 + 0.30 = 3.10.
In the debrief both interviewers admit that candidate A was the most pleasant conversation. The scorecards show why B is still ahead: no weak spot in the criteria that carry the most weight. Candidate C scored 2 on reliability, and the evidence notes show two answers about missed shifts, so the team agrees that a strong team lead score does not make up for that in this role. The decision takes ten minutes instead of an hour of competing impressions.
Fair and lawful questions
Questions should relate to the job. Asking about age, family plans, pregnancy, religion, health, origin or similar personal matters is restricted or prohibited in many countries, for example under anti-discrimination law in the EU. The exact rules differ by country, so check your local employment law and remove any question you cannot link to a criterion on the scorecard.
How it works in OrgLines
In OrgLines HR you schedule an interview for an application. It lands in the interviewer's personal calendar with reminders and on the candidate's timeline, and rescheduling updates both. The interview's scorecard is filled in through a separate link rather than inside the cabinet, so an interviewer can complete it right after the conversation.
Evaluation forms are built in the form builder with a live preview. A published form version is frozen, and changes go into a new draft, so every answer always matches the questions that were asked, which is what makes scores from different weeks comparable. Closing a candidate requires a rejection reason, and the recruiter, interview and sources reports are built from the same records; the sources report also shows a rating per channel. If a candidate does not show up, OrgLines suggests moving the application to the no-show stage, but the decision stays with a person.
Common mistakes
- Choosing criteria after the interviews, to fit the candidate you already like.
- Using a 1–5 scale without anchors, so everyone gives a 3 or a 4.
- Discussing the candidate before the scorecards are filled in, which lets the most senior voice set the scores.
- Adding a new question for one candidate that the others never got.
- Writing "good vibe" or "not a fit" as evidence instead of what was actually said.
- Letting one strong criterion override a score of 1 on a criterion the job cannot do without.
- Throwing the scorecards away after the hire, so the process can never be checked or improved.
Checklist
- Four to six criteria, taken from the job and agreed with the hiring manager.
- Weights fixed before the first interview.
- The same questions and order for every candidate.
- An anchor for every point of the scale.
- Scorecards filled in alone and straight after the interview.
- Debrief only after all scorecards are in.
- Every rejection has a recorded reason.