To shortlist candidates for a developer interview, decide the three things that would make you say no before you open a single resume, score every applicant against those three things and nothing else, and invite the top four to six. Most shortlists go wrong in the opposite order: the hiring manager reads 80 resumes, forms impressions, and then invents criteria that justify the impressions. A rubric written first is faster, fairer, and much easier to defend when a candidate asks why they were cut.
Key Takeaways
- Write the rejection criteria before reading any applications; three is the right number.
- Score each candidate 0, 1 or 2 on each criterion, using evidence from the resume or a short screen, not a feeling.
- Four to six candidates is the right size for a developer interview shortlist; more than that and the interview loop eats your month.
- A pre-vetted shortlist moves the first two steps to someone else, which is the trade a staffing partner like Ruzora offers.
Write the Rejection Criteria First
Start with the work, not the resume. What will this person ship in the first 90 days, and what would make them fail at it? For a senior backend role that might be: has run Python services in production, has worked on a Postgres schema with real load, and can explain a tradeoff in writing.
Those become your three criteria. Everything else (a famous logo, a degree, a side project in a language you like) goes in a "nice to see" column that never decides anything on its own.
Why so strict? Because criteria invented after reading are just impressions with a label on them. The candidate with the polished resume and the logo you recognize wins, and you cannot tell anyone why the quiet one with the right production work lost. In a well-known field experiment, otherwise matched resumes got different callback rates depending on the name at the top. Fixed criteria will not erase that on their own, but vague ones make it much easier.
How to Shortlist Candidates With a Simple Rubric
Give each criterion three possible scores and define them in a sentence, so that two people scoring the same resume land in the same place.
| Criterion | 0 | 1 | 2 |
|---|---|---|---|
| Python services in production | No evidence | Some production work, unclear scope | Owned a production service, can name it |
| Postgres under real load | No evidence | Used Postgres, no scale signals | Schema or query work at meaningful scale |
| Explains tradeoffs in writing | Bullet list of tools only | Some project descriptions | Clear "we chose X because Y" somewhere |
Anyone scoring 5 or 6 goes to a short screen or straight to the interview list. A 4 gets a second look. Below 4 is a no, unless you are short on candidates.
Have two people score the first ten resumes separately, then compare. If they disagree by more than one point on a row, the cell definition is too vague; rewrite it before scoring the rest. That ten-minute calibration catches more bad shortlists than anything else on this page. The same logic of fixed questions and defined scores applies once the interviews start; our post on structured vs unstructured interviews covers that side.
How Many Candidates Belong on the Shortlist
Four to six. Fewer than four and one no-show or one competing offer leaves you with nothing to compare. More than six and the loop gets long. SHRM's 2025 benchmarking found screening and interviewing alone averaging 8 to 9 days each, and every extra candidate stretches that. Slow loops lose people, as we wrote in why slow hiring loses your best candidates.
What a Pre-Vetted Shortlist Changes
The two most time-consuming steps above, finding applicants and checking the evidence, are what a staffing partner should take off your plate. At Ruzora every engineer has passed an AI interview and a graded coding assessment before they can appear on any shortlist, and you get a vetted shortlist within 72 hours of telling us what you need.
The fastest way in is to describe the role to Sol, the AI assistant on our homepage. If engineers on the bench fit, Sol shows blind profiles in the chat: seniority, experience band, skills, hours of US overlap, availability and a Strong, Good or Partial match band. Names and employers stay hidden until after a founder call and a signed agreement; then you interview the people you picked. Your job becomes comparing a handful of already-vetted profiles instead of reading 120 resumes. See Meet Sol for the full flow.
A Concrete Version
A startup posts a senior backend role and gets 120 applications.
- Pass 1, must-have check: 2 minutes each against the three criteria, yes or no. 120 x 2 = 240 minutes, 4 hours. 22 candidates survive.
- Pass 2, rubric scoring: 8 minutes each to score 0 to 2 on each criterion with a note. 22 x 8 = 176 minutes, just under 3 hours. 7 candidates score 5 or 6.
- Pass 3, 20-minute screen call: 7 x 20 = 140 minutes. One candidate has already accepted elsewhere, one cannot work the overlap hours. 5 go to the interview loop.
Total: about 9 hours of hiring-manager time to reach a shortlist of 5. That is the honest cost of doing it well yourself, and it is the number to compare against a vetted shortlist that arrives already scored.
The Honest Counterpoint
Rubrics reward people whose resumes look like the criteria. A self-taught engineer who built the exact thing you need, but described it badly, can score a 3. If your applicant pool is small (say 8 people), skip the scoring and screen everyone who clears the must-haves; you lose little time and avoid cutting a strong but unpolished candidate.
And a shortlist is only as good as the criteria. If the three must-haves are wrong because nobody agreed on what the role is for, a clean rubric will produce a clean list of the wrong people. Fix the role definition first; a hiring manager intake form helps.
Frequently Asked Questions
How many candidates should you shortlist for an interview?
Four to six for a developer role. That leaves room for one withdrawal and one weak interview while keeping the loop short enough that good candidates do not accept other offers while they wait.
How do you shortlist candidates fairly?
Write three must-have criteria before reading applications, define what a 0, 1 and 2 look like for each, and score every candidate the same way. Some teams also hide names during scoring; our post on blind hiring pros and cons covers what the evidence says about that.
Can AI shortlist candidates for you?
It can do the first pass consistently, and it does not get tired at resume 90. It also repeats whatever patterns it learned, so a human should own the criteria and check a sample of rejections. Our post on AI candidate screening goes through what it catches and misses.
The Bottom Line
Criteria first, rubric second, four to six names at the end. If you would rather start at the comparing step, tell Sol what you are hiring for and see who on our vetted bench fits, or request a shortlist directly.
Roberto Espinoza is CEO of Ruzora, which helps US startups hire pre-vetted senior LATAM engineers, with a vetted shortlist in 72 hours. See available engineers.
