How to choose a penetration testing provider

Almost every provider promises the same things: experienced testers, a thorough methodology, a clear report, a retest. The promises are cheap and nearly identical, so the decision cannot be made by reading them. What separates providers is who actually sits at the keyboard, how much of your system their time genuinely covers, and whether what they hand back is specific enough for your engineers to act on. All three are knowable before you sign, if you ask directly.

What you are really buying

A penetration test is not a product with a specification. It is a quantity of skilled attention, pointed at a system you care about, for a bounded period. The tooling is commodity — the same proxies, the same frameworks, the same public research — and the report template is a formatting decision. What differs wildly between providers at similar prices is the judgement of the person deciding where to look.

The difference between a tool alert and a useful finding is demonstrated consequence. A reflected parameter needs investigation: where does the value land, under which account, and what can an attacker reach? That investigation may use human-led, automated or combined techniques. Compare the evidence and the application context, rather than assuming the delivery mode proves the result.

The same applies to a workflow replayed to apply a discount twice or an identifier accepted across a tenant boundary. The assessment needs to exercise the relevant business rules and permissions. Ask how the provider does that, how it records coverage and who investigates unexpected results. Our comparison of automated and specialist testing explains how to evaluate both approaches.

The questions that separate providers

Ask these in writing and compare the answers, not the brochures.

Who performs the test, and what have they actually done? The most load-bearing question, and the one most often answered with a company biography rather than a person. Ask for the named individuals on your engagement, how long each has been testing systems like yours, and what they have published, found or built. Then two follow-ups: if a named tester becomes unavailable, who replaces them and at what level; and is a junior's work reviewed by someone who re-examines the system, not just the prose?

Is this manual testing, or validated scanning? Both are legitimate; the problem is only ever one being sold as the other. Ask what proportion of the effort is a tester working by hand, what the tools are used for, and how findings are validated. A good answer is specific about the division of labour: automation for enumeration and the repetitive classes where recall matters; humans for authorisation logic, business logic, chained attacks and anything where impact must be demonstrated rather than asserted. A weak answer describes a platform and never says who reads the output. Our account of that division is on our methodology page; expect theirs in comparable detail.

How much of the application, and how many roles, will genuinely be covered? Coverage is the quietest variable in the purchase. Ask whether every authenticated area is covered or a sample, and if a sample, how it is chosen and how you will be told what was left out. Then ask about roles — not "is authorisation tested" but how many will be exercised, and whether horizontal access control (one user reaching another user's data at the same privilege level) is tested as well as vertical. Access-control work grows with roles multiplied by sensitive operations, not added to them, so a proposal treating a six-role application like a two-role one has decided to sample.

What does the report actually contain? Do not accept a contents page. Ask to read a complete deliverable — anonymised, findings written out in full. Every provider calls its reports clear and actionable; only the document shows whether findings carry evidence, whether remediation advice is specific to your stack or generic to the class, whether severity reflects business impact or only a vector string, and whether anything is written for the people who sign it off. We publish ours for this reason — you can read a complete sample report without talking to anyone — and you should expect the same from anybody asking for your budget.

Will findings come with reproduction steps? A finding your engineers cannot reproduce is a dispute waiting to happen. Ask whether each includes the exact request, the preconditions, the account used and the observable result — enough that a developer can reproduce it without emailing the tester. A screenshot of a scanner's verdict is not evidence of exploitability; a captured request and response is. It is also the best proxy for whether a finding was validated, because a tester who exploited something can always show you how.

Is a retest included, and what does it cover? Ask whether retesting is in the price, how long you have to use it, whether it covers all findings or only the severe ones, and what you receive. A letter listing each finding as fixed, partially fixed or unresolved is worth having; a verbal "looks fine" is not. Ask what that letter claims: an auditor reads it literally.

What happens if something critical is found mid-test? Ask for the mechanism, not the reassurance. Who is contacted, by which channel, within how long, and what does the tester do next — stop, document and continue, or establish full impact? Ask whether you get a written summary before the report, because an exploitable authentication bypass needs fixing immediately, and whether they will tell you if they find evidence someone else got there first.

Will the people who tested it debrief your engineers? A technical debrief with the actual testers is where much of an engagement's value is realised, and it is frequently the first thing cut. Ask whether it is included, who attends, and whether questions during remediation are chargeable. The practical test: can your lead developer get a straight answer from the person who wrote the finding, without a commercial conversation?

Who owns and retains the data? Testing generates credentials, extracted records, screenshots of real data, and a report that is a roadmap to your weaknesses. Ask where it is stored, who inside the provider can read it, how long it is kept, and whether your report will be used as a sales sample. Then the questions your own obligations force: will data be held in jurisdictions your contracts allow; will they sign your NDA and data processing agreement before scoping detail is shared; will any subcontractor touch the engagement? Ours are on our trust page, which is the form the answer should take — written down, not described on a call.

Reading a proposal

A scoped proposal and a templated one look alike at a glance. The tell is specificity about your system. A scoped proposal names your applications, counts your roles, mentions the authentication mechanism you described, identifies which integrations can and cannot be exercised safely, and states what it excludes. A templated one describes a methodology in general terms, lists vulnerability classes applicable to any web application, and derives effort from a count of assets without engaging with what makes them different.

Three signals are worth looking for. Does it state what is out of scope as explicitly as what is in? Does it make its assumptions explicit, so an incorrect one becomes a conversation rather than a surprise? Does it specify what you must provide and by when — accounts per role, seeded data, allowlisting, a named contact?

The scoping conversation beforehand is at least as diagnostic. A thorough one feels like an interrogation of your architecture rather than a qualification call: the authorisation model and where it is enforced, how many roles are genuinely distinct rather than cosmetic variants, how tenant isolation is implemented, which workflows involve money or regulated decisions, and what you would consider the worst realistic outcome. Expect some pushback: a provider who says your scope is too large for the depth you want is doing you a favour. One who accepts a half-described scope without a clarifying question has decided what to sell you irrespective of what you need. You can see the shape of a scoped engagement in how we describe penetration testing; use it as a comparison, not a template.

Credentials, and what they do and do not prove

Certifications are useful and routinely over-read. The practical ones — OSCP, OSWE, OSEP, the OSCE³ set, CREST's registered and certified tester examinations and their equivalents — are earned in timed, hands-on exams against live targets. Passing one is evidence that a particular individual could, on a particular day, do a particular class of work unaided, which is more than a multiple-choice credential tells you.

What they do not evidence is company quality. A certification attaches to a person, and people are assigned to engagements: a firm can truthfully say its engineers hold strong certifications while the tester on your project holds none of them. Nor do they speak to what most often goes wrong — whether scoping was honest, whether coverage matched the proposal, whether the report is usable, whether anyone senior reviewed the work.

Company-level accreditation does exist in some schemes, and where it does it assesses the organisation — processes, quality assurance, data handling — rather than only its staff. That is a different kind of assurance, and in some sectors a hard requirement. If a scheme membership is mandatory for you, verify it on the scheme's own register rather than a supplier's footer, and check what it covers: an accreditation's scope is often narrower than its logo implies.

To be plain about our own position: PXLSec's engineers hold individual practical certifications, including OSCE³, OSWE, OSEP, CRTO and CREST CRT. We do not hold company-level accreditation under any scheme and do not claim to. If your procurement rules require an accredited supplier, we are not eligible, so say so early rather than discover it at contract stage. Weigh instead what accreditation stands proxy for: the named people, their verifiable work, and a deliverable you can read first.

Price, and what a cheap quote is usually buying

Cheap quotes are rarely dishonest. They are usually a different service with the same name, and the difference is almost always in one of six places — each leaving a trace.

Automated scanning sold as testing. The proposal describes a platform or dashboard more than people; findings are promised as a count rather than written narratives; the methodology lists vulnerability classes rather than activities. Ask what a tester does by hand, and for one finding a scanner could not have produced. A firm doing manual work answers immediately.

Junior testers with no senior review. The proposal cites combined company experience and never individuals; the rate is uniform regardless of who works on what; nothing mentions quality assurance. Ask for names and years, and who reviews the work. Juniors on a well-chosen scope, supervised, is how good testers are made; juniors working unsupervised on your authorisation model is not.

No retest. It is absent, priced as an option, or described without a scope — the commonest way to make a quote look competitive, because the omission only hurts later, when you need evidence the findings are closed. Search the proposal for the word, and check which findings it covers and what document you receive.

Templated reports. Remediation advice generic to the vulnerability rather than specific to your stack; an executive summary that could be about anyone; thin evidence. Reading a full sample is the highest-yield hour here: if the fix advice would suit any application anywhere, you are buying a document.

Undisclosed offshoring or subcontracting. The contracting entity, the stated delivery location and the testers' location are not clearly the same, and the proposal is silent on who performs the work. This may be entirely fine; not being told is not, because it affects your data processing obligations and sometimes your regulator. Ask where testers sit, where data is stored and whether any part is subcontracted — then check the contract agrees.

Scope quietly narrowed to fit. The subtlest of the six. The quote matches your budget because the work was trimmed to match it, and the trimming lives in omissions — fewer roles than your application has, "the main application" where you described three, APIs folded into the web line, a sample instead of full coverage. Read the scope section against your own asset and role list, and ask what was excluded and why. A provider who narrows scope deliberately and tells you is managing a real constraint; one who narrows it silently has sold you the appearance of coverage.

None of this means the most expensive proposal is best. Price also reflects overheads, commission and brand, none of which test your application. Compare price against named people, stated coverage and a deliverable you have read.

Red flags

• No named testers, or names that change between the proposal and the kick-off call without explanation.
• Unwillingness to show a complete sample deliverable. A contents page is not a sample.
• Guaranteed outcomes — a promised number of findings, a promise of none, or a promise that you will pass an audit. None are within a tester's gift.
• A certificate of compliance as the headline deliverable, especially where it claims more than the testing supports.
• Findings without reproduction steps or captured evidence.
• Severity ratings with no business reasoning, or a sample report where everything is high.
• Pressure tactics: a quote expiring this week, a discount for signing today, urgency manufactured around your audit date.
• Tooling named as the methodology, with no account of what humans do.
• Silence on data handling — retention, location, access, destruction — or reluctance to sign an NDA and DPA before scoping.
• Reluctance to put the testers in a room with your engineers.

Questions to ask your shortlist

Send these to each provider unedited. They are answerable in a short email by anyone who has thought about them.

1. Who will perform this test, how long has each been testing systems like ours, and who reviews their findings before issue?
2. What proportion of the effort is manual, what are the tools used for, and how are findings validated?
3. How many distinct roles will be exercised, and is horizontal as well as vertical access control tested?
4. Will the whole application be covered or a sample, and how will we be told what was not reached?
5. May we read a complete anonymised report before deciding?
6. Will every finding include reproduction steps, the account used and captured evidence?
7. Is a retest included in this price? Which findings does it cover, over what window, and what document do we receive?
8. If you find something critical mid-test, who do you contact, how quickly, and what next?
9. Is a technical debrief with the testers included, and can our developers question them during remediation?
10. Where is our data stored, who can access it, when is it destroyed, and is any part of this engagement subcontracted?
11. Will you sign our NDA and DPA before we share scoping detail?
12. What is out of scope, what are you assuming about our environment, and what happens if the application proves larger than scoped?
13. Can you describe one finding from a past engagement that a scanner could not have produced?
14. What would make you tell us this engagement is the wrong thing to buy?

The last is the most informative. Every honest provider has a version of the answer — that a code review would serve you better, that your scope is too wide for the depth you want, or that you should fix what you already know about first.

After you choose

How much value you get is now largely in your hands. Testing time is finite, and it is most often wasted on your side of the fence.

Prepare access before the start date, not during it. Accounts for every role, verified working, with seeded data that makes authorisation testing meaningful — an empty account proves nothing about isolation. Allowlisting where required, a decision on whether protective controls stay on, and a stable environment: if the target falls over nightly, testing becomes an exercise in waiting. A day lost to access is a day of testing you paid for.

Name one contact with authority to answer questions. Someone who can confirm whether a behaviour is intended, authorise a slightly wider action, and find the right engineer quickly — reachable on a channel the testers use, and not on leave during the window. Questions that wait become assumptions, and assumptions become findings that are wrong in one direction or another.

Plan remediation capacity before the report lands. Reports sit not because nobody cares but because nobody has time allocated. Reserve capacity for the weeks after delivery, decide who triages findings and settles severity disagreements, and agree how fixes will be verified internally. Expect to disagree with a rating occasionally; a good provider will change it where you are right.

Use the debrief properly. Put the developers who own the affected code in the room, not only management. Ask how each finding was discovered, because that generalises further than the finding itself, and what the testers would look at next given more time — usually the most valuable unbilled thing in the engagement.

Use the retest, on time. Fix, verify internally, then submit within the window — unused retests are the most commonly wasted part of a test. Check the closure letter says what your auditor needs, and no more.

Do this well and the test produces fixed systems and engineers who recognise the pattern next time. Do it badly and it produces a PDF. The provider influences which of those you get; so do you.