By requirement
The AI Act names the attacks. Someone has to test for them.
Article 15 requires high-risk AI systems to be resilient against specific attacks — data poisoning, model poisoning, adversarial examples and confidentiality attacks. It names them. It does not say who demonstrates that resilience, or how.
What it asks for
What Article 15 actually says
Article 15(1) requires high-risk AI systems to be designed and developed so that they achieve an appropriate level of accuracy, robustness and cybersecurity, and to perform consistently in those respects throughout their lifecycle. The obligation sits with the provider, and it is a design-and-development obligation rather than a one-off test at the end.
Article 15(5) is the one that reads like a testing scope. High-risk AI systems must be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. The text then names the attack classes it has in mind: data poisoning, which manipulates training data; model poisoning, which compromises pre-trained components; adversarial examples or model evasion, which are inputs crafted to cause a mistake; and confidentiality attacks or model flaws. That is an unusually specific list for a regulation, and it is a long way from a conventional application penetration test.
Article 15(4) covers robustness more broadly: systems must be as resilient as possible to errors, faults and inconsistencies, including those arising from interaction with people or other systems. For systems that keep learning after deployment, it requires the risk of biased outputs feeding back into future inputs to be eliminated or reduced as far as possible.
Art. 15(1) – Accuracy, robustness and cybersecurity
High-risk AI systems must achieve an appropriate level of accuracy, robustness and cybersecurity and perform consistently in those respects throughout their lifecycle. A design-and-development obligation on the provider, not a certificate obtained once.
Art. 15(4) – Resilience to errors and feedback loops
Resilience to errors, faults and inconsistencies, including those arising from interaction with natural persons or other systems. Systems that continue to learn after deployment must reduce as far as possible the risk of biased outputs influencing future inputs.
Art. 15(5) – Resilience to attack
Resilience against unauthorised third parties altering the system's use, outputs or performance by exploiting vulnerabilities. The text names data poisoning, model poisoning, adversarial examples or model evasion, and confidentiality attacks or model flaws.
Art. 50 – Transparency obligations
Separate from Article 15 and already applicable: disclosure where people interact directly with an AI system, marking of synthetic content, and disclosure for emotion recognition, biometric categorisation and deepfakes.
Annex III – Which systems are high-risk
Biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration and asylum, and administration of justice. The AI Omnibus left the substance of Annex III unchanged and added no new categories.
Chapter V – General-purpose AI models
A separate obligation set for providers of general-purpose AI models, in application since 2 August 2025, with a later compliance date for models already on the market before then.
What to test
What this means for a testing scope
The AI administration surface
The shortest route to altering a system's outputs: who can write the prompt configuration, reach the pipeline's storage credentials, operate the model lifecycle or trigger ingestion. An attacker who can rewrite the instructions does not need an adversarial example.
The training and ingestion path
Where data poisoning lives. What can reach the corpus, who can trigger a retrain, and whether an ingested document can carry instructions that survive into the model's behaviour.
Adversarial inputs and model evasion
Against the system as deployed, judged by what they actually reach rather than by whether the model can be made to say something unwanted.
Confidentiality of data and model
Whether one tenant's conversations, documents, embeddings or generated artefacts are reachable by another, and whether the model or its surrounding services leak training data, system instructions or tool definitions.
What you receive
What a conformity assessment will want to see, and what we produce
Findings mapped to the named attack classes
A report that maps findings to the attacks Article 15(5) names, so a reviewer can see each was considered rather than inferred from a generic application test.
Evidence and a reproduction path per finding
With a specific fix, in a form an engineer can act on and a reviewer can follow.
A stated scope, and stated exclusions
A resilience claim without its boundary is not a resilience claim. What was excluded is written down, and so is what that means for the assurance.
A retest, confirmed in writing
Confirming what was remediated and closed — the part that turns a finding list into a demonstrable improvement.
Common pitfalls
Where this goes wrong
Mistakes that cost time at audit, and how to avoid them.
Assuming an application pentest covers Article 15
It does not. Poisoning, evasion and model confidentiality are not in a standard web or API methodology, and a report that never mentions them will not read as evidence of Article 15 resilience.
Treating prompt injection as the whole of it
Injection is in scope, but Article 15(5) is about altering use, outputs or performance by exploiting vulnerabilities — and in practice the administration surface around the model is a shorter path than the model itself.
Working to the old deadline
The AI Omnibus, in force on 27 July 2026, moved the Annex III high-risk obligations to 2 December 2027. A great deal of published guidance still shows the original August 2026 date.
Expecting a test report to be a conformity assessment
It is not, and no testing firm can issue one. The report is an input to a process somebody else runs.
Who decides what counts
PXLSec is an independent offensive-security firm, not an auditor, assessor, certification body or regulator. We deliver the testing and the evidence; your auditor, assessor or competent authority decides what they accept. This page is general guidance, not legal advice.
FAQ
Questions we're often asked
Does a penetration test make us AI Act compliant?
No. Compliance is a conformity assessment process, and testing is one input to it. We produce security evidence about your system: what an attacker can do to its outputs, its data path and its administration surface, with the evidence and the fixes. Whoever runs your conformity assessment decides what they accept. We are a testing firm, not a notified body, an auditor or a certification body.
Is our AI system high-risk?
That is a legal determination about your use case, not a technical one, and it turns on Annex III and on whether the system is a safety component of a regulated product. We do not make that call and you should not take it from a testing firm. If you are high-risk, Article 15 applies; if you are not, the attacks it names are still the ones that will be used against you.
The deadline moved — do we still need to do anything?
The Annex III high-risk obligations moved to 2 December 2027, so there is more time than the original August 2026 date suggested. The substance did not change, and neither did the attacks. If your AI feature is already in production, the exposure is live regardless of when the obligation bites.
We use a third-party model. Is that in scope?
The model you call is not yours to authorise testing against, and anyone offering to test someone else's hosted model cannot lawfully deliver it. What is in scope is everything you built around it: the prompt configuration, the data you feed it, the tools you let an agent call, the isolation between your tenants, and who can change any of that.
Do you test for bias or safety alignment?
No, and we would rather say so than take the work. Bias, toxicity and alignment evaluation is a different discipline with different methods and different people. We test security: whether someone can alter the system's use, outputs or performance by exploiting it. Article 15 covers both accuracy and cybersecurity; we are on the cybersecurity side of it.
Can you map findings to Article 15 for us?
Yes. Where a requirement drives the engagement we scope against the clause that applies and write the report so it can be handed over as it is, with findings mapped to the attack classes the text names. That mapping is ours as testers; the judgement about whether it satisfies your obligation is not.
Services that deliver this
Discuss your testing scope.
Tell us what you want to protect. A senior tester will help you agree the scope and next steps. We usually reply within one business day.
Confidential from the first conversation
We can sign your NDA before you share sensitive details. Your enquiry is covered by our privacy policy.
How we protect your information · Privacy policy