B-52 · Agentic Penetration Testing Platform

The six penetration testing phases, and why QA is the last one

Discovery, planning, scanning, exploitation, reporting, QA. The first five will be familiar from any testing methodology you have read. The sixth is the one worth stopping on. Where it sits is the point, not what it contains.

A quality gate placed mid-pipeline can only hold work back, because there is no report yet to check. Ours runs on the finished report, so it can pull one apart and send the findings back to the phase that produced them before it reaches you.

The run, in order

01 Discovery What is there, inside the scope you signed off
02 Planning A mindmap, and the test cases this application needs
03 Scanning Candidates, none of which is a finding yet
04 Exploitation Proved, or dropped. Nothing survives on a guess
05 Reporting The report, and the findings in your dashboard
06 QA A gate that can send the finished report back

What runs inside the phases

Where the work inside those phases came from

Security Brigade has been CERT-In empanelled since 2008, and every assessment the firm has run since it started in 2006 was worked inside Lemon, our own assessment platform. It is where 6,700+ assessments of test cases, vulnerabilities and threat models accumulated, and that is the corpus behind our models.

The B-52 harness was then written to run the four practices a senior Security Brigade auditor works through on an application assessment. They are listed below in the order they feed each other, because the fourth is worth very little without the three before it.

The four auditor practices the B-52 harness reproduces
Link	Alone	In sequence
1 Mindmap creation	A picture of the application: its entry points, its roles, and the paths between them.	The map every generated test case is derived from, so the checks belong to this application.
2 Test-case generation	A list of checks, which is the thing a scanner already ships with.	Checks written against what this application does, instead of a standard list run at whatever is in front of it.
3 Comprehensive JavaScript analysis	A large amount of client-side code, read and not skimmed.	Routes, parameters and endpoints the interface never links to, added to the map before any of it is tested.
4 Functional flow analysis	A note of the multi-step flows and the state they carry.	Those flows taken out of order, and taken by the wrong role, which is where the logic breaks if it is going to.

Each practice on its own, and what it contributes once the one before it has run

What autonomy adds on top of those four practices

The generated test set is worked through in full, instead of being sampled down to fit the days that were sold.
The four practices run the same way on every target in scope, whoever scoped the engagement and whenever it was scoped.
A run can happen on the cadence your releases actually ship at, which no schedule of booked engagements supports.

Why the benchmark reads the way it does

B-52 was run in parallel with Security Brigade’s own expert assessment team, on the same targets, and the two sets of findings were counted against each other. Measured against the combined set (everything either party found, counted once), B-52 reached 90–95%, and it surfaced issues the human team did not.

Comparable coverage, different blind spots. The reason it comes that close is the four practices above: it was built from the work of the people it was measured against.

How the benchmark was measured

Phase by phase

Six phases, and each one has to close before the next opens

Read scanning and exploitation as the separate phases they are. Scanning produces candidates; exploitation either proves one or drops it. Nothing arrives in your report on a scanner’s say-so.

01 Discovery

The surface as it stands today, not as the documentation describes it

What runs: B-52 establishes what is actually reachable: hosts, applications, endpoints, and the roles that can get to each of them. On an application this is also where the client-side code is read, which is how routes nothing on the interface links to end up in scope for testing.
Before it moves on: The authorised scope is the boundary. B-52 tests what you signed off and nothing outside it, however interesting the thing outside it looks.

02 Planning

A mindmap of the target, and the test cases this application needs

What runs: Two of the four auditor practices sit here. The mindmap is drawn from what discovery found, and the test set is generated from the mindmap, so the checks are derived from what this application does.
Before it moves on: The test set belongs to this target. Nothing enters the run because it appears on a generic list of things worth trying.

03 Scanning

Candidates. Nothing that leaves this phase is a finding

What runs: The planned test cases are worked through against the live target, and what comes back is a set of candidates: places where the behaviour suggests something is wrong.
Before it moves on: A candidate is a lead and is treated as one. Nothing produced in this phase is counted, reported or shown to you.

04 Exploitation

Each candidate is either proved or dropped

What runs: B-52 attempts the exploit. A candidate that survives carries the request that triggered it, the response that came back, and the steps that reproduce it. A candidate that does not survive is dropped, not downgraded to a note.
Where the run stops for you: Three actions need your written approval before B-52 takes them: a destructive or state-changing action against a production system, persistence or lateral movement past the host it first landed on, and anything that touches live credentials or real customer data.

05 Reporting

The proven findings, written up and sitting in your dashboard

What runs: The report is produced, and the same findings appear in the dashboard with their evidence attached, so your engineers work from the artefact instead of a summary of it.
What varies by model: In the fully autonomous model, findings pass an independent automated cross-check before they are reported. In the expert-verified model, a senior auditor verifies every one of them before anything reaches you.

06 QA

The report is written. This is where it gets taken apart

What runs: The finished report goes through a quality gate before it reaches you. A finding that does not hold is sent back to the phase that produced it, and the report is corrected and not annotated.
Why it is sixth and not fourth: A gate placed mid-pipeline can only hold work back, because the report does not exist yet. Placing it after reporting means the thing being checked is the finished artefact (the report as you would read it) and not merely the work behind it.

The three approval gates in the exploitation panel are the whole of what B-52 stops for. Everything above them runs without asking; everything at them waits until you have said yes in writing.

The autonomy boundary in full

Where a person can step in

The same six phases in every delivery model

The delivery models differ by where the human sits, never by what is tested. So the table below is the whole of the difference between them, phase by phase, and the first column is mostly empty because that is what the fully autonomous model is.

Where a person can act in each phase, under each delivery model
Phase	Fully autonomous	Autonomous, expert verified	Human led
Discovery	You authorise the scope and the targets before the run starts, and that is the last thing this model asks of you unless the run reaches one of the three approval gates.	The same authorisation, on the same terms. This model adds its auditor at reporting.	A senior auditor is running the engagement, and this phase with it.
Planning	No person acts, and nothing is waiting on an approval at this point.	Unchanged from the fully autonomous model.	The auditor is directing the run, with B-52 underneath it.
Scanning	No person acts, and nothing produced here is shown to you.	Also unchanged. The auditor has not entered the run yet.	The auditor is directing the run, with B-52 underneath it.
Exploitation	The run stops for your written approval at each of the three gates, and carries on once you have given it.	The same three gates, and the same written approval from you.	The same three gates, with the auditor at the controls.
Reporting	No person acts. Findings pass an independent automated cross-check before they are reported.	A senior auditor verifies every finding before anything reaches you. That verification is the whole difference between this model and the fully autonomous one.	The auditor writes the report.
QA	No person acts; the gate runs against the report.	Nothing further is asked of you once the findings have been verified.	A senior auditor is running the engagement, and this phase with it.

Where a person can act in each phase, under each delivery model

Discovery

Fully autonomous: You authorise the scope and the targets before the run starts, and that is the last thing this model asks of you unless the run reaches one of the three approval gates.
Autonomous, expert verified: The same authorisation, on the same terms. This model adds its auditor at reporting.
Human led: A senior auditor is running the engagement, and this phase with it.

Planning

Fully autonomous: No person acts, and nothing is waiting on an approval at this point.
Autonomous, expert verified: Unchanged from the fully autonomous model.
Human led: The auditor is directing the run, with B-52 underneath it.

Scanning

Fully autonomous: No person acts, and nothing produced here is shown to you.
Autonomous, expert verified: Also unchanged. The auditor has not entered the run yet.
Human led: The auditor is directing the run, with B-52 underneath it.

Exploitation

Fully autonomous: The run stops for your written approval at each of the three gates, and carries on once you have given it.
Autonomous, expert verified: The same three gates, and the same written approval from you.
Human led: The same three gates, with the auditor at the controls.

Reporting

Fully autonomous: No person acts. Findings pass an independent automated cross-check before they are reported.
Autonomous, expert verified: A senior auditor verifies every finding before anything reaches you. That verification is the whole difference between this model and the fully autonomous one.
Human led: The auditor writes the report.

Fully autonomous: No person acts; the gate runs against the report.
Autonomous, expert verified: Nothing further is asked of you once the findings have been verified.
Human led: A senior auditor is running the engagement, and this phase with it.

One action is common to all three: you authorise the scope and the targets before anything starts. After that the fully autonomous model asks nothing further of you except at the three approval gates, and those are gates you set the terms of when you sign the scope.

Persistence, where it is approved, is established by the platform. There is no point in the run where a human operator takes over to do the part that supposedly needs a person, because that part is in the platform as well.

If the assessment is going to a regulator, take the expert-verified or the human-led model. The empanelled auditor’s involvement is what makes the output signable, and CERT-In empanelment is a condition of the testing itself and not only of the vendor who sells it.

Choosing a delivery model

The same phases, different work

The same six phases on a web app, an AD estate and a mobile binary

A web application, an Active Directory estate and a compiled mobile app all run through the same six phases, and the work inside those phases has very little in common. A mobile assessment is the clearest illustration, because it starts from a file, not from a URL.

B-52 decompiles and analyses APK and IPA binaries itself, so the input to a mobile run is the release build you already ship, and there is no source code to hand over.

What a mobile run does in each phase

Three things a mobile run has to deal with first

Hardened release builds Obfuscated production binaries are the normal input, not the exception that gets scoped out of the engagement.
Static and dynamic, both platforms Android and iOS, with the static and the dynamic work done on the same build and with no source required for either.
Pinning and runtime protections Certificate pinning and runtime application self-protection are handled by the platform. Getting past them is the work that has to happen before the application behind them can be tested at all.

Every finding leaves with the exploit that produced it

The request, the response, and the steps that reproduce it, in all three delivery models, whichever of the six phases the finding came out of. Anything that could not be reproduced was dropped at exploitation and not written up with a caveat attached to it.

The platform in full Request a Scoping Call