VA, Pentest, BAS, Red Team: What Each One Actually Tests

Security assessment budgets tend to follow two patterns. The first is compliance-driven: buy a penetration test annually because a regulation or contract requires one. The second is reactive: something happened, leadership wants to know if there are other vulnerabilities like it, so buy an assessment.

Neither pattern connects assessment spending to the specific security questions the organization needs answered. The CDM fixes this by mapping each assessment type to a specific column. Once you understand what column an assessment tests, you can sequence your assessment investments based on which columns in your program are weakest and which questions are most important to answer.

The four assessment types and their columns

Each major assessment type answers a fundamentally different question about a different column in the CDM:

Vulnerability Assessment tests the Identify column. The question it answers: where are we inherently vulnerable? VA tools scan systems for known vulnerabilities, misconfigurations, missing patches, and weak credential configurations. The output is a list of findings ranked by severity and exploitability.

What it doesn’t test: whether those vulnerabilities are mitigated by compensating controls, whether your detection would catch an attempt to exploit them, or whether your response capability is functional. A high-severity finding in a VA report tells you the vulnerability exists. It tells you nothing about whether an attacker could successfully exploit it in your environment.

Penetration Testing tests the Protect column. The question it answers: are those vulnerabilities actually exploitable given the defenses we’ve put in place? A penetration tester attempts to exploit findings from vulnerability assessment and any additional weaknesses they discover, specifically to determine whether the protective controls in place prevent exploitation.

The penetration test answers a question that the VA cannot: “yes, this vulnerability is present, but does our network segmentation, our WAF, or our authentication controls prevent an attacker from reaching it?”

What it doesn’t test: whether your detection would catch the activity if a real attacker attempted the same exploitation. Most standard penetration tests are not designed to test detection; they’re scoped to demonstrate exploitation path. If your pentest scope says “stay under the radar,” you’re testing something slightly different, but still not explicitly testing your detection coverage.

Breach and Attack Simulation tests the Detect column. BAS platforms (Cymulate, AttackIQ, SafeBreach, and several others) continuously or periodically execute simulated attack behaviors and measure whether your security controls detect and alert on them. The question BAS answers: for the attacks we’re simulating, do we log the event and generate an alert at the moment just before an analyst would need to act?

This is a specific and valuable test. It tells you not whether the vulnerability exists (VA) or whether it’s exploitable (pentest), but whether your detection stack is generating the right signal when attack behavior occurs. A BAS run that shows 40% detection coverage across a simulated attack scenario is telling you that 60% of those behaviors would occur silently in your environment.

Red Team Exercise tests the Respond column. A red team doesn’t just exploit vulnerabilities and doesn’t just test whether alerts fire: a red team runs a sustained adversary simulation to determine whether a realistic attack chain, executed with operational security, would result in a detection event that gets escalated to an incident and triggers a real response action. The question is: does our response actually fire when a skilled adversary is operating in our environment?

Red team is expensive, requires significant planning, and consumes significant analyst time during the exercise. It answers a question the other three methods can’t: not “are we vulnerable” or “can we be exploited” or “do we detect attacks” but “when a real adversary operates in our environment with realistic tradecraft, do we catch them and respond effectively?”

Why sequencing matters

The four assessment types aren’t interchangeable and aren’t all equally valuable at every stage of program maturity. The CDM logic suggests a natural sequence.

You need Identify coverage before a pentest tells you anything useful. If you don’t have a reasonably complete picture of your attack surface, a pentest that successfully exploits a known vulnerability in a system you were already aware of doesn’t produce new information. And a pentest that fails to find anything exploitable in the systems you gave them might miss significant risk in systems you didn’t know existed. Vulnerability assessment first, penetration testing second.

You need Protect coverage before BAS tells you anything useful at the Detect level. BAS measures whether detection fires when attacks occur. If your protective controls are so weak that attacks succeed trivially, the BAS output will be dominated by “this worked because the protection failed” rather than “this worked because the detection missed it.” Those are different problems with different fixes. Sort out your protective controls before you start measuring detection.

Red team is a test of full program integration. It’s most valuable when you have baseline capability across Identify, Protect, and Detect: you know your attack surface, you have defenses in place, you have detection coverage. The red team tests whether all of that actually coheres into a functioning defensive capability against a sophisticated adversary. Running a red team on an organization with immature detection capability produces a report that says “we weren’t detected” without telling you whether that’s because detection is broken or because the red team evaded detection. Those are different answers.

In practice: most organizations over-invest in penetration testing relative to their maturity level and under-invest in everything else. An annual pentest at an organization with no tuned SIEM, no detection use cases, and no incident response playbooks is $50,000-$200,000 spent on Protect column testing while the Detect and Respond columns are empty.

BAS as a continuous measurement tool

One underutilized property of BAS platforms: they can run continuously, which means they can measure detection coverage as a metric over time rather than as a point-in-time assessment.

This connects directly to the CDM scorecard approach in Article 6. If you’re scoring your Detect cells in a quarterly posture review, BAS data is a direct input: what percentage of the attack behaviors we care about are currently generating alerts? Did that percentage go up or down since the last measurement?

BAS also provides a direct feedback loop between your Protect team and your Detect team. When BAS shows that a specific attack technique is not being detected, the investigation often reveals either that the attack bypassed a protective control that should have caught it (Protect gap) or that the detection rule for that technique doesn’t exist or is misconfigured (Detect gap). That distinction matters for where you direct remediation effort.

The major BAS platforms map their simulation library to MITRE ATT&CK technique IDs. Combined with the CDM, this creates a three-layer view: CDM cell shows you where in the matrix you have a detection gap, ATT&CK technique ID shows you exactly what behavior you’re missing, and BAS data shows you the current detection rate. That’s a specific enough picture to write a meaningful detection engineering backlog.

Red team considerations for APJ

Red team engagements in Japan and the broader APJ market have some specific characteristics worth noting.

Engagement scoping tends to be more conservative than in North American or European markets. Japanese enterprises in particular often scope red team engagements to specific systems rather than full adversary simulation across the enterprise, which limits the value of the exercise for testing organizational response. A red team that’s confined to a network segment can tell you about detection coverage in that segment; it can’t tell you whether your incident response escalation path works when the incident spans multiple systems and teams.

The more useful engagement model for organizations that are new to red teaming is a purple team exercise: structured collaboration between an adversary simulation team and the defending security team, with explicit knowledge sharing about what’s being simulated and real-time feedback on detection and response. Purple team is not a test of whether you can catch an adversary; it’s a training exercise to improve detection coverage and response procedures. For organizations with immature Detect and Respond capability, purple team produces more actionable output than a black-box red team engagement.

JFSA guidance on security assessments (cybersecurity assessment requirements for financial institutions under the FSA Guidelines for Supervision) includes requirements for periodic security testing but doesn’t prescribe specific methodologies in detail. The CDM mapping of assessment types to columns provides a useful structure for demonstrating to regulators that your assessment program systematically tests each layer of the defensive architecture rather than just checking an annual pentest box.

The question each assessment can’t answer

Knowing what each assessment type doesn’t test is as important as knowing what it does.

VA tells you where vulnerabilities exist. It doesn’t tell you which of those vulnerabilities are actually reachable by an attacker given your network architecture, or which are mitigated by compensating controls.

Pentest tells you which vulnerabilities are exploitable in the specific scope tested. It doesn’t tell you whether an attacker with a different entry point, different timing, or different tooling would be detected.

BAS tells you whether your current detection stack generates alerts for specific simulated techniques. It doesn’t tell you whether your analysts act on those alerts, whether the escalation process works, or whether your response would be effective.

Red team tells you whether a skilled adversary operating in your environment triggers detection and response. It doesn’t continuously monitor your detection coverage or tell you whether the result generalizes to different adversary tooling or techniques.

The full picture requires all four, sequenced appropriately. In practice, most organizations will run VA continuously, pentest annually (or more frequently for high-risk systems), BAS continuously or quarterly, and red team biennially or after major architecture changes.

That’s a significant assessment investment, and not every organization has the budget for all four simultaneously. The CDM prioritization logic applies: invest in assessment capability in the columns where you have the least operational confidence, and sequence accordingly.

Next: Article 9 — Zero Trust Is Not a Product: What the Matrix Reveals