What evidence should you ask for when someone claims AI experience?
8 minute read. Updated 2026-08-08.
Ask for one project described in enough depth that the candidate can go several layers deeper on any question: the constraint, the decision, the failure, the fix. Treat vague phrases like 'AI transformation' and unexplained titles as warning signs. A genuine NDA covers a client's identity and data, not the reasoning behind a decision, so ask for that reasoning with names removed. Call the person who commissioned the work and ask what broke, not whether they would rehire.
A claim on a CV is worth nothing on its own
Anyone can write machine learning engineer or AI transformation lead on a CV. Titles are self-reported, unregulated, and, as it happens, barely used consistently even in the advertised job market itself. This is also why a marketplace built on ratings alone cannot solve the problem: a rating tells you a past client was satisfied, not that the underlying work was any good.
A specialist charging near the top of the market, close to £788 a day, the 90th percentile advertised for the broad AI category over the six months to August 2026 according to IT Jobs Watch, should be able to produce evidence to match that rate. If they cannot, the rate is the only thing you have confirmed.
This is not about catching people out. Most people who claim AI experience have some, at some level. The question that matters commercially is whether the level and kind of experience matches the problem you are paying them to solve, and evidence is the only way to find that out before the engagement starts rather than during it.
What good evidence looks like
Good evidence is specific and falsifiable. Ask the candidate to walk through one project in detail: what the system was for, what constraints it worked under, what they tried that failed, and what they changed as a result. Someone who did the work can go several layers deeper on any question you ask. Someone who did not runs out of detail quickly and retreats to generalities.
Architecture diagrams they can draw and explain from memory, evaluation results they can defend against an obvious objection, a clear account of a decision they got wrong and why, and a specific figure they can justify under questioning, are all strong signs. A candidate who volunteers a mistake and explains the fix is showing you more than one who claims everything worked first time.
Ask about scale too, but treat the answer as one data point rather than the whole picture. A specialist who worked on a system processing a handful of requests a day and one who worked on a system under heavy load faced different problems, and a strong candidate will say so unprompted rather than letting you assume the harder version.
If the candidate cannot answer questions about scale at all, treat that gap the same way as a vague answer on architecture: as missing evidence rather than as neutral information.
What weak evidence looks like
Vague language is the biggest tell. Phrases like 'led the GenAI transformation' or 'delivered end-to-end AI solutions' describe a job title, not a piece of work. If a candidate cannot name the specific technical decision they made and why they made it that way rather than another way, they are describing someone else's project, or a project that mostly existed on a slide.
Certificates and course completions are evidence of study, not evidence of delivery. They are fine as a supporting detail. Treated as the main evidence for an AI specialist role, they tell you the candidate can pass a test, which is a different skill to shipping a working system.
Buzzword density is a reasonable proxy for weak evidence, though not proof of it. A candidate who reaches for 'transformative', 'end-to-end' or 'strategic' rather than naming the specific model, dataset shape, or failure they dealt with is either early in their career or padding a thin project. Either is worth knowing before you hire, not after.
Handling genuine NDA work without letting it become an excuse
Real NDAs exist and a competent specialist respects them. What a genuine NDA covers is the client's identity, their data, and commercially sensitive detail. It does not cover the reasoning behind a technical decision, the trade-offs considered, or the shape of a failure and its fix. A specialist can describe all of that without naming the client or revealing a single confidential number.
A useful test: ask them to describe the problem, the constraints, and the decision they made, with the client's name and specific figures removed. Someone who has genuinely done the work does this fluently, because the technical reasoning was never the confidential part. Someone who reaches for 'I can't discuss any of it' at the first question is either overcautious to the point of being unhelpful, or has less to say than the CV implies.
Be alert to the difference between an NDA and a preference not to discuss a poor outcome. Both produce the same sentence, 'I can't go into that', but only one is a legal constraint. Asking a follow-up question about the reasoning, rather than the client, usually reveals which one you are dealing with.
If a candidate cannot pass this test after several attempts, across several different past projects, treat the NDA explanation as exhausted and weight the interview accordingly. A single project genuinely locked down by an unusually strict NDA is plausible. An entire career that cannot be discussed in any technical detail is not.
References should ask about failure, not fit
A reference call is only useful if it asks a different question to the interview. Skip 'would you work with them again' and ask what they actually built, what broke, and how they responded when it did. Ask the referee to describe one decision the candidate made that they disagreed with at the time.
This tells you how the candidate handles being wrong, which is the situation that actually matters on a live engagement. It is worth calling the person who commissioned the work directly rather than a colleague who observed it from the side, even if that call is harder to arrange.
It is also worth asking the referee what the candidate did not do, or refused to do, and why. A specialist who pushed back on an unrealistic deadline or a risky shortcut is showing exactly the judgement you are trying to verify, and it rarely appears anywhere on a CV.
Build a short, specific scenario into the process
The most reliable single piece of evidence you can generate yourself is a short scenario built from a real, anonymised version of your own problem. Give the candidate limited information, ask them to describe how they would approach it, and see where they ask for more detail before answering.
A specialist who asks sharp, specific questions before giving an answer is showing you how they actually work. One who gives a confident, generic answer to a vague scenario is showing you how they interview, which is not the same thing.
Keep the scenario short enough to run inside an interview rather than as homework; 20 minutes is usually enough to see how someone thinks. The goal is not to solve your actual problem for free, it is to watch the shape of their reasoning under a small amount of real constraint.
What to do about it
- Ask for one project described in detail rather than a list of technologies used.
- Treat vague phrases like 'AI transformation' as a warning sign, not a credential.
- Accept NDA limits on client identity and data, not on describing the reasoning behind a decision.
- Call the person who commissioned the work and ask what broke, not whether they'd rehire.
- Build a short scenario from your own anonymised problem instead of relying on a generic take-home test.
Questions people also ask
What if a candidate genuinely can't share any project detail due to NDA?
A genuine NDA covers the client's identity and their confidential data, not the reasoning behind a technical decision. Ask the candidate to describe the problem, the constraints and the trade-off they chose, with names and figures removed. Anyone who did the work can do this without breaching anything. If they cannot discuss even anonymised reasoning, treat that as a gap in the evidence rather than proof of confidentiality.
Are certificates and courses worth anything as evidence?
They show the candidate has studied a subject, which is worth something but is not evidence of delivery. Weight them low compared with a detailed account of a real project, and never accept a certificate as a substitute for it. A candidate with strong project evidence and no certificates is a safer hire than one with several certificates and no detailed project to discuss.
How technical do I need to be to judge the evidence myself?
You need to be able to ask 'why that, and not the alternative' and notice whether the answer is specific or generic. You do not need to judge the technical merits of the answer yourself. Depth and consistency across follow-up questions are the signal, not whether you personally agree with the technical choice.
Should the reference check happen before or after the technical interview?
After. The interview tells you what the candidate can articulate under mild pressure. The reference tells you what happened when the pressure was real and nobody was performing for an audience. Running the reference call last means you go in with specific claims from the interview to test against what the referee actually remembers.
Where the figures come from
Every rate and salary quoted in this article is a median or percentile of figures advertised in UK job postings over the six months to 8 August 2026. They are not rates paid, and the gap widens at the top of a range.
- IT Jobs Watch, UK contract rates, 6 months to 8 August 2026, read 2026-08-08.
- IT Jobs Watch, UK permanent salaries, 6 months to 8 August 2026, read 2026-08-08.