How to vet a candidate's experience during an interview

November 16, 2026

Regardless of the role you're interviewing for, one question you'll almost always want to ask is: "Tell me about your past experience."

However, this question can be quite vague, and you'll get all kinds of answers, which might not be what you're after.

I think what you really want when you ask this question is for the candidate to:

  • Demonstrate a complex project or product they worked on
  • Show how close they were to the real work
  • Show you that their experience is relevant to the role you're hiring for

If this is what you're after, here's how I would approach this question.

How to ask the question

  1. Set up the question. Ask them to describe one initiative they led or were heavily involved in. Give two ground rules up front:
    • It must be customer or external-facing, not internal. Internal projects usually have far more leeway
    • It can't be a one-dimensional project. It must have both breadth and depth
  2. Ask them to frame the logistics first:
    • When it took place: this helps you frame how to evaluate what they did (e.g. building an agentic harness in 2025 is very different than building one in 2026)
    • How long it ran: this helps you gauge if what they're about to tell you sounds reasonable based on your expectations of speed
    • Who was involved, and their contributions vs others: this clarifies how they fit into the puzzle, and what work they actually did
  3. Let them tell the high-level story without interrupting
  4. Dig in where it gets hard. As the story unfolds, find the moments of complexity or impact and ask them to double-click. Push for exact details about the customer, the approach and the team dynamic.
  5. Ask for reflection: "If you did this all over again, what would you do differently, and why?"

How to evaluate the answer

Grade the candidate on a 1-4 scale for each of the following high-level dimensions. It's important to keep the grading rubric high-level, as specific details can trap a good candidate into "missing a checkbox".

  1. Ownership. Were they truly as involved as they say?
    • Their ownership is clear and separate from the team's, and it fits the team size and timeline
  2. Depth. Can they recall exact details when you double-click?
    • Numbers, decisions, and why the alternatives lost. Vague answers on the parts they claim to own are a red flag
  3. Breadth. Do they grasp the whole situation, not just their slice?
    • They can explain the customer, the approach and the team dynamic
  4. Reflection. Is their answer on what they'd change specific and honest?
    • A real lesson with a reason. A lot of people take this opportunity to humble brag instead

📧 johnny@johnnyji.com