Red flags in a penetration test proposal

A proposal is a document written to win work, which makes it a poor description of the work and an excellent sample of how the supplier thinks. Read it for what it commits to rather than what it claims. Most of the signals below have innocent explanations — the point is not to disqualify anyone, but to know which question turns a claim into a commitment.

A quote with no scoping conversation

You sent a two-line enquiry and received a priced proposal next morning. Nobody asked how many roles your application has, whether it is multi-tenant, or whether a usable test environment exists.

Why it matters. Effort tracks the structure of the thing being tested, not its headline description. A quote produced without that understanding is a guess, and a guess that proves low gets absorbed by reducing depth. You will not see that happen; you will see a report with a plausible finding count.

Sometimes innocent. If you supplied a thorough scoping document, a diagram and a role matrix, a fast quote shows competence. A clearly labelled indicative range, offered early so you can set a budget, is also helpful — provided the label is there.

Ask. "What did you assume about our environment to produce this number, and which assumptions would change the price if wrong?" A supplier who has thought about it lists assumptions readily. One who cannot name any has not made them consciously.

Effort that cannot cover the stated scope

The proposal lists several applications, an API, a cloud environment and an internal network, then allocates effort that would be tight for one of them. Arithmetic, not opinion.

Why it matters. Underscoping is the commonest way a test disappoints without anyone lying. The methodology may be real and the testers good; there was simply no time to traverse an authorisation matrix across six roles, learn a pricing engine and enumerate a network. What arrives is a sample presented as an assessment, and the report rarely says which parts were sampled.

Sometimes innocent. Ten endpoints behind one gateway with one authorisation model genuinely cost less than two unrelated applications. A lean figure may be well judged — the test is whether the supplier can explain the grouping.

Ask. "How is the effort distributed across the scope, and what will you not have time to do?" The second half is the useful half. Every time-boxed engagement has a boundary; a supplier who states theirs has planned, and one who insists there is none is selling a fiction.

"Automated and manual testing", doing a great deal of work in one sentence

The phrase is near-universal and almost meaningless. In an honest proposal it describes a workflow: tooling for coverage and known issues, humans for logic, authorisation and chaining. In a dishonest one, a scan with somebody reading the output.

Why it matters. Scanners find known vulnerable versions, missing headers, misconfigurations and injection in obvious parameters. They do not find broken object-level authorisation between two customers, a workflow replayable to apply a discount twice, or a password-reset sequence that yields another account. Those need someone to understand what the application is for, then violate that intent.

Sometimes innocent. Automation belongs in any good methodology — nobody enumerates a large estate by hand, and a tester who refuses tooling is wasting your money. The question is only which findings come from where.

Ask. "Which classes of finding do you expect from tooling, and which from manual testing? Give me one example from the manual half that no scanner would produce." That separates the two camps immediately, because the second camp has no example to give.

No named methodology

A proposal should say what the testing systematically covers, by reference to a recognised framework appropriate to the target — a web application standard, a mobile guide, a cloud benchmark — ideally mapped onto your scope.

Why it matters. Methodology is how a test becomes repeatable and coverage auditable. Without one, the engagement reflects whatever that tester happened to think of, which varies by person, by day, and by how much time is left. It also makes this year's report incomparable with last year's.

Sometimes innocent. Frameworks are a floor, not a ceiling. Senior testers routinely go beyond published methodology, and objective-driven work is legitimately organised around goals instead of coverage — the objectives should then be stated with equivalent precision.

Ask. "Which methodology sections apply to our scope, and which do not?" Someone who works to a framework answers in its language. Compare the answer with what the provider publishes about its own testing methodology, since a public description is harder to improvise.

No sample report

The report is the product; everything else is logistics. A supplier who will not show you a redacted or demonstration report is asking you to buy the one thing you cannot inspect first.

Why it matters. A sample tells you in ten minutes what the proposal never will: whether findings carry reproduction steps or only descriptions; whether evidence is real request and response material or a screenshot of a scanner panel; whether remediation advice is specific to your stack; and whether the coverage section admits what was not reached.

Sometimes innocent. Client reports are confidential, and hesitating over a real one is correct. A provider should instead hold a purpose-built or thoroughly redacted example; PXLSec publishes a sample report for this reason.

Ask. "May I see a full sample, including the detail pages rather than the summary?" Then read one finding end to end and ask whether your own engineer could act on it without a meeting.

No retest, or a retest that costs extra

Retest terms are either in the proposal or they are not. If they are not, the default is that you pay again to learn whether your fixes worked.

Why it matters. A finding is closed when someone who understands the attack confirms it no longer works, not when a developer says so. Fixes fail often: the filter covers one parameter and not its sibling, the authorisation check reaches the endpoint but not its batch variant, the patch lands on one of three instances. A test without verification leaves you holding beliefs about your own systems.

Sometimes innocent. Excluding retest can be the honest structure — if remediation will run for a year, a separately scoped revisit beats a short window nobody uses. The red flag is silence, not exclusion.

Ask. "Is retest included; for which findings; how many passes; within what window; and does it test bypasses of the fix or only the original issue?" The last clause matters most, since a hurried patch frequently creates a near-identical second route.

Findings promised without reproduction steps

"Detailed descriptions of each vulnerability with risk ratings and recommendations" describes a document. "Step-by-step reproduction, with the evidence required to replicate independently" describes a deliverable.

Why it matters. A finding your team cannot reproduce becomes a dispute rather than a ticket. Developers reasonably decline to change code on an assertion, and without steps the argument often ends in a downgrade by exhaustion. Steps also protect you at retest and in front of an auditor — the only evidence that something was demonstrated rather than inferred from a version banner.

Sometimes innocent. Some findings resist reduction to steps: a design weakness, an architectural observation, a race condition that reproduces intermittently. Those belong in the report with their uncertainty stated plainly. It is missing steps for ordinary mechanical findings that signal a problem.

Ask. "Will every exploitable finding include reproduction steps and raw evidence, and may I have that as a contractual deliverable?" If yes, put it in the statement of work, where it is worth something.

A scope that lists assets but no roles

The scope names applications, hostnames and API paths and says nothing about user roles, tenants or test accounts — the most reliable sign of a proposal written by someone who has not tested an application recently.

Why it matters. Access control is not a feature you check once; it is a matrix you traverse. For every sensitive operation a tester must establish which roles may perform it and attempt it as each role that may not — then, for each role, attempt to reach data belonging to another instance of that same role: another user, another team, another tenant. Two different tests, and where the serious findings in most business applications live.

Sometimes innocent. An unauthenticated external infrastructure assessment has no roles, and neither does a review of a public marketing estate. For anything with a login, the omission is substantive.

Ask. "How many accounts and tenants do you need, at which roles, and what should be in them?" A supplier who means to test authorisation has a precise, slightly demanding answer, including distinguishable seeded data. One who says "an admin login is fine" has told you the plan.

"Our team of experts", and nothing more specific

You are buying attention and judgement, and a proposal describing the testers only in collective, flattering terms withholds the main variable.

Why it matters. The spread between a competent senior tester and a junior working a checklist is wider than any other factor, and it is invisible in the proposal and barely visible in the report. Sales attention concentrates on the pitch while delivery is staffed by whoever is free that week. Nobody need be deceiving you; the proposal simply never made a commitment that could be broken.

Sometimes innocent. Providers reasonably avoid naming individuals months ahead, and mixed-seniority teams are a good structure — juniors do excellent work under review, and it is how the profession reproduces itself. What you want committed is the standard, the review and the accountable lead, not the names.

Ask. "Who performs this work, at what seniority, with what experience of our stack — and who is accountable for report quality?" Then: "Is any part subcontracted?" Specialist subcontracting is often right, but learn of it now rather than from a footnote.

Boilerplate that mentions the wrong technology, or another client

The proposal describes testing your Kubernetes estate when you run virtual machines, references a mobile application you do not have, or still carries another organisation's name in a header, a footer or a file property.

Why it matters. Two problems. Nobody read the document before sending it, and the same care will reach your report. Worse, another client's name in your proposal means your name will appear in someone else's — and a firm that leaks client identities through sloppy document handling is holding a report describing how to compromise you.

Sometimes innocent. Templates are sensible, and methodology sections should be reused rather than rewritten. The issue is boilerplate that contradicts your environment, which suggests the scope section was not written for you either.

Ask. Point at it: "this mentions a technology we do not use — which parts were written for us?" Then ask how client reports are stored and transmitted, and compare the answer with what the provider publishes about its own security practices.

Compliance claims the provider cannot make

The engagement is said to make you compliant, satisfy an auditor, or tick a regulatory box outright. Compliance language runs loosest in proposals because it is where buyers are most anxious.

Why it matters. A test produces evidence that a control was assessed. Whether that evidence satisfies a framework depends on scope, on the assessor, and on your wider control environment. No provider can promise an audit outcome they do not control, and the failure surfaces late: the audit arrives, the scope does not cover what the standard required, and the work is redone.

Sometimes innocent. Many providers understand specific frameworks well and scope precisely against them, which is valuable. "This scope meets the testing requirement in clause X, and here is how the report maps to it" is a legitimate claim. "This will make you compliant" is a different sentence.

Ask. "Which specific requirement does this evidence, and what else does the framework require that this engagement does not cover?" The last part is the test — a provider who knows the standard will happily say where their work stops.

A report turnaround that implies no review

Testing finishes Friday; the final report is promised Monday. Welcome — but worth examining.

Why it matters. A report sent without peer review goes out with one tester's conclusions unchallenged. Review is where severity gets corrected, a missing reproduction step is caught, and a second reader asks whether a finding was proven or merely suspected. That takes time, and a turnaround with no slack has usually removed it. The opposite extreme is also a flag: a report six weeks later is written from cooling notes, and the week after testing is when you most need it.

Sometimes innocent. Good teams write as they test, so a short gap can reflect discipline. Interim findings delivered immediately, with a reviewed final report following, is best practice rather than haste.

Ask. "Who reviews the report before it reaches me, and what does that review check? Will I get interim findings during testing?" A supplier with a real process describes steps and named roles. One without describes care.

The review checklist

Work down this list with the proposal open. Each item is answerable from the document, or by one question.

1. Scoping. Which assumptions produced this price, and which would change it?
2. Effort. How is it distributed across the scope, and what will there not be time to do?
3. Automation. Which findings come from tooling, which from manual work — plus one example no scanner could produce?
4. Methodology. Which sections of a named framework apply here, and which do not?
5. Sample report. May I see a full one, detail pages included — and could my engineer act on a finding unaided?
6. Retest. Included? Which findings, how many passes, what window, bypasses tested?
7. Evidence. Reproduction steps and raw evidence for every exploitable finding, contractually?
8. Roles. How many accounts and tenants, at which roles, with what data in them?
9. People. Who tests, at what seniority, and who owns report quality?
10. Subcontracting. Any part subcontracted, to whom, under what data terms?
11. Boilerplate. Which parts were written for us? Whose name is in the properties?
12. Compliance. Which requirement does this evidence, and what does the framework still require?
13. Turnaround. Who reviews the report, what does review check, and are there interim findings?
14. Data. Where will our report be stored, for how long, and who can read it?

Two patterns are worth keeping. The useful questions all ask for a boundary — what is not covered, what there is not time for, where the methodology stops, what the framework still requires. Capable suppliers answer those comfortably; the rest reach for reassurance. And the answers belong in the statement of work, not your notes from the call: a proposal is marketing, the contract is what you bought. Reading a provider's public service description beside its proposal, and noting where the two diverge, is a cheap final check.