Every senior engineer now claims they use AI tools. The resume says Copilot, Cursor, and Claude, the same way it used to say "team player." The claim is free, so it tells you nothing. What tells you something is watching how the person actually works with these tools on a real problem, because the gap between an engineer who uses AI with judgment and one who leans on it without any is now one of the biggest differences in output you can hire for.
Key Takeaways
- Do not screen for tool names on a resume. Screen for how the engineer uses them live.
- The signal is judgment: knowing when to trust an AI suggestion and when to throw it out.
- Test the failure case, not the happy path. Watch what they do when the model is confidently wrong.
- We built an AI-tooling stage into our vetting for exactly this reason.
Why the Resume Line Is Worthless
Listing "AI tools" on a resume in 2026 is like listing "email" in 2005. Everyone has them open. The question is whether the engineer ships more because of them or ships subtle bugs faster because of them. A weak engineer with an AI assistant produces plausible-looking code they do not fully understand, which is worse than slow correct code, because it passes review and breaks later. The tool amplifies whatever judgment is already there. You are hiring the judgment, not the subscription.
What to Actually Test
Give the candidate a real, slightly ambiguous task and let them use whatever tools they want, out loud. Then watch for a few things.
| Watch for | Strong signal | Weak signal |
|---|---|---|
| Prompting | Frames the problem, gives context | Pastes the ticket and hopes |
| Reviewing output | Reads it, questions it, tests it | Accepts the first suggestion |
| Knowing the limits | Drops AI for the parts it is bad at | Uses it for everything equally |
| Debugging | Uses AI to explore, not to guess | Loops on the same wrong fix |
The most important moment is when the AI is confidently wrong, which it will be. A strong engineer notices the smell, checks the assumption, and corrects course. A weak one keeps prompting variations of the same broken idea because the tool sounds sure of itself.
A Concrete Version
Hand two candidates the same task: add rate limiting to an endpoint, using any tools they like. The first pastes the ticket into the assistant, takes the first middleware it suggests, and moves on. It works in the demo and silently shares state across processes, which will fail the day you scale to two replicas. The second asks the assistant for options, notices the suggested approach assumes a single process, asks about your deployment, and picks the version that holds under real load. Same tools, same ten minutes. One of them just shipped a production incident and the other caught it, and the only difference you could see was whether they trusted the output or interrogated it.
The Honest Counterpoint
You can overweight this and screen out a great engineer who simply prefers to write more of the code themselves. AI fluency is a real signal, but it is not the only one, and a brilliant systems thinker who uses the tools lightly is still a brilliant systems thinker. The point is to reward the best judgment, not the heaviest AI use, and to make sure that when the person does reach for AI, they can tell good output from confident nonsense. Test for that, and do not turn it into a purity test about how much of the code a human typed.
Frequently Asked Questions
Should I require experience with a specific AI tool?
No. The tools change every few months and they are similar to learn. Screen for judgment using AI in general, not fluency in one vendor's product.
How is this different from a normal coding interview?
A normal coding interview watches someone write code. This watches someone decide which AI output to trust, which is the newer and now more predictive skill. Let them use the tools, and grade the decisions.
What is the single best test?
Give a task where the obvious AI answer is subtly wrong, and see whether they catch it. Catching a confident wrong answer is the whole skill in one moment.
The Bottom Line
AI tools do not make a weak engineer strong. They make a strong engineer faster and a careless one dangerous. Stop screening for tool names and start watching how someone works when the model is wrong. That is the difference that shows up in your codebase six months later, and it is the thing worth testing for. See how we test it in the five-stage vetting process, and why it matters in why AI-proficient engineers matter. See available engineers.
Roberto Espinoza is CEO of Ruzora, which helps US startups hire pre-vetted senior LATAM engineers, with a vetted shortlist in 72 hours. See available engineers.
