Attacks a system deliberately to find how it breaks before someone else does.
What separates them
Anyone confident can produce a jailbreak; the discipline is turning that into a finding someone else can reproduce, rank and actually fix.
Ask these
01
Show me, in your own words, what a finding write-up looks like once you've broken a system. Not the exploit itself, the structure of the report.
Tests whether output is reproducible and actionable rather than a party trick.
A strong answer
They describe reproduction steps and a severity and likelihood assessment. They also describe a suggested fix or mitigation, and how they checked the finding wasn't a one-off before writing it up.
The confident failure
They describe the exploit vividly and enthusiastically but the report structure is vague, with nothing on severity, reproducibility or how an engineer would actually act on it.
02
How do you decide what to prioritise when you've found several different ways to break a system and only one can be fixed before launch?
Tests judgement rather than attack creativity alone.
A strong answer
They describe a concrete method, such as ranking by likely real world exploitation combined with severity of harm, and give an example where the most impressive looking finding was not the one they prioritised.
The confident failure
They say all findings get reported and it's up to engineering to prioritise, which avoids the judgement call, or rank purely by how impressive the exploit looked.
03
Tell me about a system where you found nothing wrong. How did you know that meant it was safe, rather than that you hadn't tried hard enough?
Tests calibration and honesty about the limits of red teaming, a common blind spot in this work.
A strong answer
They describe what categories of attack they tried and which they didn't get to, and say plainly that a clean result is evidence against the approaches they tried, not proof of overall safety.
The confident failure
They present a clean result as a confident endorsement of the system's safety with no mention of what wasn't tested.
04
How do you keep red teaming useful once a team has seen your previous attacks and started defending against them specifically?
Tests whether they treat this as an ongoing adversarial process rather than a one-off audit.
A strong answer
They describe adapting the approach as the target evolves, for example rotating attack categories or bringing in fresh testers, and checking whether a fix actually generalises rather than just patches the reported case.
The confident failure
They describe running the same test suite on a regular schedule as sufficient, without addressing that a known suite stops finding anything new once the team has seen it.
What we ask when assessing for the register
Harder, and answerable only by somebody who has done the work. Published because a question that stops working when it is known was never testing anything.
01
Walk me through a finding you produced that engineering disputed. What was the disagreement and how did you settle it?
Only genuine practitioners have had a finding contested; tests rigour under pushback.
A strong answer
They describe a specific technical disagreement, for example about whether the exploit would recur in practice, and how they reproduced it under different conditions or gathered further evidence to settle it.
The confident failure
They say findings were generally accepted without pushback, or that disagreements were settled by seniority rather than by evidence.
02
Describe a case where you had to demonstrate that a vulnerability was not a one-off, that it would recur reliably. How did you establish that?
Distinguishes a repeatable methodology from having got lucky once.
A strong answer
They describe a specific reproduction protocol, running the attack across multiple sessions or prompt variants and reporting a hit rate rather than a single instance.
The confident failure
They describe the vulnerability as clearly severe without describing any reproduction testing, implying the severity was self-evident from a single occurrence.
Ask these whatever the discipline
Tell me about something you built that failed in production. What broke, how did you find out, and what did you change?
What would you refuse to do on this project, and what would you tell me instead?
How would you know, three months in, that this was not working?
What is the part of your own work that you are least confident about?
Relevant here, and none of them is evidence on its own. What each does and does not prove is set out in full on the certifications page.
AI Security Management, AAISM, ISACA. Hands-on adversarial capability. Managing a security programme and breaking a model are different jobs.
Offensive Security Certified Professional, OSCP, OffSec. Model red teaming. It is a systems and network qualification. Somebody holding it can probably learn to attack a model; it is not evidence that they have.