If a finding isn't reproducible, it isn't finished

The most useful question to ask a prospective testing firm isn't about their tools or their certifications. It's: "Can I see an anonymised report?" And the thing to look for in that report is whether a developer who wasn't on the call could reproduce every finding from the page alone. If not, the report isn't finished.

Why reproducibility is the whole game

A finding exists to be fixed. Fixing it means an engineer has to understand it, confirm it, change code or configuration, and verify the change worked. Every one of those steps depends on being able to reproduce the issue. A report that says "SQL injection present, severity High" and stops there has handed the engineer a mystery, not a fix.

The anatomy of a reproducible finding

• Exact location. The precise endpoint, parameter, request or screen — not "the application."
• Preconditions. What role, what state, what data needs to exist first.
• Steps. The actual requests or actions, in order, that trigger the issue — copy-pasteable.
• Evidence. The request/response pair or screenshot that proves it happened.
• Impact. What it means for the business, in plain language.
• Fix. A concrete remediation, ideally with a code or configuration example.

What it tells you about the firm

Reproducible reporting is hard to fake. It means the tester actually understood the issue well enough to explain it to someone else — which means they actually exploited it, rather than copying a scanner's one-line description. Vague reports are often a sign that nobody manually verified the finding at all.

It's also why we re-test. If we can't reproduce a finding ourselves after you've fixed it, we can't honestly tell you it's closed. Reproducibility runs through the whole engagement, from the first finding to the closure letter.

The executive summary is part of it too

Reproducibility isn’t only for engineers. Leadership has to “reproduce” the risk decision — understand what’s at stake well enough to prioritise and fund the fix — and they can’t do that from a wall of technical detail. A good report carries a short executive summary that states the overall risk, the handful of themes behind the findings, and the two or three things to fix first, in plain language. If the first two pages don’t let a non-technical decision-maker act, the report isn’t finished for them either.

Findings as data, not just prose

A genuinely useful report also ships its findings as data — a CSV or JSON export that drops straight into your issue tracker. Reproducibility and trackability go together: a finding your team can’t import is a finding that quietly slips between the report and the backlog. The deliverable isn’t the document; it’s the fixes that get made because of it.