Pentest and Red Team Harnesses

Module S03 · Course 2A

90 min · Kill Chain State · Plan Correction · Sandbox Inversion · Report Generation

Prerequisite: S00, S01, S02

S03.1 — Kill Chain State Management

S02's engagement memory was a flat surface — independent findings. A red team kill chain is a sequence with dependencies: foothold requires initial access, lateral movement requires foothold, exfiltration requires privilege.

The state is not a set of findings; it is a graph of dependent objectives.

MITRE ATT&CK as the state machine

Each ATT&CK technique is a node. The kill chain is a path. Node states:

unexplored → attempted → succeeded / failed / out_of_scope

Explicit prerequisites and edges. The harness must answer: "why am I allowed to attempt lateral movement now?"

Field	Purpose
`tactic` / `technique`	ATT&CK labels (e.g. T1190 Exploit Public-Facing App)
`prerequisites`	node IDs that must be `succeeded`
`evidence_refs`	linkage to the evidence chain (S02.3)

Attack graph vs. linear plan

Linear plan
Single objective, single technique, high confidence. A one-shot CVE validation.

Attack graph
Multi-objective, multiple techniques per objective, dependencies. Any engagement >1 day.

Switch mid-engagement: start linear, promote to graph the moment a step fails or a second objective appears. A linear plan is a degenerate single-chain graph.

Anti-pattern: collapsing a dense graph back to a linear plan "for simplicity." Throws away the alternative paths you'll need when the primary technique fails.

Why general harnesses lose the chain

Two specific failure modes for multi-stage kill chains:

Intermediate state loss
Initial access succeeds → context compacts 3 calls later → agent forgets foothold exists → lateral movement prerequisite check fails.

Cross-session loss
Day 1 achieves foothold. Day 2 fresh session has no memory → re-attempts initial access (noise, detection risk) or proceeds without the dependency.

Cure: the attack graph is persisted OUTSIDE the context window, queried before every step.

S03.2 — Plan Correction Under Adversarial Conditions

An "error" in an offensive engagement is an exploit that didn't land. The correct response is not retry — it is re-plan. This is the single biggest differentiator between a script and a harness.

Source: RedTeamLLM (Brown et al., 2024) — the four-step plan correction loop.

The four-step plan correction loop

1. DETECT — distinguish hard failure (patched, rejected) from soft (timeout, rate limit)

2. RE-HYPOTHESIZE — what ELSE satisfies this tactic? Query knowledge base, exclude_failed=True

3. RE-PLAN — mark old node FAILED, add NEW unexplored node (same prerequisites)

4. RETRY — execute the NEW node, not the old one

Key invariant: failed techniques are removed from the candidate set → forces genuine substitution, not variation. Cure for exploit lock-in.

Generalize vs. specialize

Strategy	When	Tradeoff
Specialize	High-confidence intel (exact version, confirmed config)	High impact if it works; fails immediately if intel is wrong
Generalize	Low-confidence intel, OR specialized technique already failed	Reliable, noisier, broad pattern

RedTeamLLM ablation: harnesses with a generalize-on-failure fallback complete 40% more objectives. In adversarial environments, specialized intel is frequently wrong.

S03.3 — The Sandbox Inversion Problem

Course 1: sandbox CONTAINS the agent to protect the host.
Offensive: agent must reach OUT to attack the target — but a compromised agent (prompt injection, S01.3) can attack YOU.

This is the load-bearing sub-section. The offensive sandbox inverts the Course 1 model.

Dual containment — two independent boundaries

1. Outbound scope enforcement
Egress proxy + firewall ACL generated from scope.json. Agent may connect ONLY to declared in-scope targets. Enforced at the NETWORK layer, not just middleware.

2. Inbound isolation
No inbound routes from target network to container or host. Reverse shells/callbacks terminate in the container.

Both required. Middleware checks intent; network layer enforces outcome. If one is bypassed, the other catches it.

Credential quarantine + the metadata trap

Engagement-discovered credentials (DB passwords, API keys, hashes) must never touch host credential stores (SSH agent, keychain, ~/.aws/credentials).

Cure: a separate encrypted vault inside the container, own key, destroyed on teardown. Host stores not mounted at all.

The metadata trap: deny 169.254.169.254 at the proxy. A compromised agent reaching the cloud metadata endpoint steals the HOST's cloud credentials. Always denied.

Verify both directions independently

A harness that has not verified containment is making an untested safety claim. Verification = attempt the forbidden thing, observe the block.

Direction	Test
Outbound	From container, connect to an out-of-scope canary → REFUSED. Connect to in-scope → SUCCEEDS.
Inbound	From target segment, connect back to host/management → REFUSED. Only control plane traverses the boundary.

S03.4 — Report Generation as a Harness Output

The deliverable is not the attack. The client cannot invoice a reverse shell. The deliverable is the report.

Report generation is a first-class harness output, designed from the start — generated from structured data accumulated through the engagement.

CVSS is always a DRAFT

CVSS scoring is a known LLM weakness. The model produces plausible vector strings — and gets them wrong by 2+ points regularly. Overstates on scary descriptions, understates on subtle paths.

Policy: CVSS is always auto-scored, always flagged "DRAFT — pending analyst review", always routed to human review before the client. Never trained on as ground truth.

Report pipeline

Extract findings (one per succeeded node) + evidence_refs

CVSS draft (flagged) + CWE mapping (deterministic lookup)

Remediation: OWASP control (lookup) + target-specific detail (model)

Executive summary LAST — grounded in the specific finding set

HTML
interactive, clickable evidence

PDF + JSON
static signable + machine-readable

One source of truth, three renderings. Separate report data from engagement data → two sources → they drift.

What you take into S04

The red team harness is operational: attack graph state, plan correction, dual containment, client-ready reporting.

S04 applies these patterns to CTF harness engineering: shorter engagements, known-flag objectives, the specific harness tweaks that win competitions.