From Checklist Fatigue to a Real Security Scorecard

Security checklists have a bad reputation and most of it is deserved. Not because the questions are wrong, but because the output is usually a single number or a pass/fail that obscures more than it reveals. “We’re 73% compliant with CIS Controls” tells you almost nothing about where you’re exposed or what to fix first.

The CDM offers a better approach. Map your checklist questions to cells, score each cell independently, and you get a heat map of your actual posture rather than a compliance average that hides the distribution.

What’s wrong with undifferentiated scoring

Take the CIS Controls v8. There are 18 controls, 153 safeguards. If you score compliance as a single percentage, a perfect score on Controls 1-10 combined with zero implementation of Controls 11-18 looks like 56% overall. That’s technically accurate and operationally misleading: you might have excellent asset management and terrible incident response capability, and the aggregate score doesn’t tell you which.

The same problem applies to ISO 27001 Annex A, NIST SP 800-53 control families, FISC security guidelines, and every other checklist-based assessment framework. Compliance percentages aggregate across the entire control set. The distribution within that aggregate is where the actual risk information lives.

Yu demonstrated this in the “Reloaded” presentation using US Cyber Consequences Unit checklist questions. Each question maps to a CDM cell. Count the questions per cell, score how many your organization satisfies, and you get a fraction for each cell: 19/34, 23/25, 6/9. Convert to percentages and put them in the matrix. The result is a visual heat map of where you’re strong and where you’re exposed.

The value isn’t the numbers themselves. It’s that the distribution across cells is immediately visible.

Mapping controls to cells

Most major control frameworks map reasonably cleanly to CDM cells. The mapping isn’t always one-to-one, and some controls span multiple cells, but the exercise produces something more useful than the original framework’s own reporting structure.

CIS Controls v8: Yu’s “Reloaded” presentation includes a detailed mapping of CIS Top 20 safeguards to CDM cells. The Devices row is densely populated, particularly in Identify and Protect, because CIS Controls weight endpoint and asset management heavily. The Users row is thin in Detect and Respond because most checklist frameworks underweight behavioral analytics and insider threat. The Applications row is uneven: strong in Protect (secure configuration, application hardening) and weak in Detect.

NIST SP 800-53: The control families map fairly naturally. AC (Access Control) is Protect across Users and Networks. AU (Audit and Accountability) is Detect across all rows. IR (Incident Response) is Respond. CA (Assessment, Authorization, Monitoring) spans Identify and Detect. CP (Contingency Planning) is Recover.

FISC Security Guidelines (Japan FSI): The 14th edition guidelines cover endpoint security, network security, application security, data protection, access management, and incident response. These map to CDM cells with reasonable precision. The FISC guidelines weight Devices/Protect and Networks/Protect heavily, which reflects the historical threat model of the Japanese financial sector. Detect and Respond controls are present but less prescriptive, which is consistent with the finding in most CDM exercises that Detect and Respond cells are less mature than Protect cells.

The mapping exercise itself has value independent of the scoring. Going through a control framework and asking “which CDM cell does this belong to?” forces you to think about what the control is actually doing rather than checking it off as a line item.

Building the scorecard

The mechanics are simple. For each cell:

1. Collect all the checklist questions or control requirements that map to that cell.
2. Assess how many your organization satisfies, and at what fidelity. A control that’s documented but not implemented is not the same as one that’s implemented and tested.
3. Record the fraction and the percentage.
4. Optionally, weight by criticality: a question about encryption of data at rest might carry more weight than a question about banner suppression on web servers.

Across 25 cells, you now have a heat map. Color the cells: green for cells above your target threshold, yellow for cells in a warning range, red for cells below a minimum acceptable level. The visual immediately communicates what a percentage score can’t: which asset classes are well-covered, which operational functions are weak, and whether the weakness pattern suggests a systematic gap or isolated holes.

A systematic gap in the Detect column across all asset rows suggests you have a detection program problem: you’re protecting well but you’d be slow to notice a breach. A systematic gap in the Users row across all functions suggests your human risk program is underdeveloped. A gap concentrated in Data/Respond suggests you’d struggle to scope and contain a data breach effectively.

These are actionable observations. A 67% overall compliance score is not.

Fidelity matters more than presence

One trap in this exercise: scoring a control as satisfied because the tool exists rather than because the capability is operational.

“We have a SIEM” is not the same as “our SIEM has tuned detection rules covering our priority use cases and analysts working the alert queue.” The first is a tool present answer. The second is a capability present answer.

In CDM terms: the cell isn’t covered by the presence of a tool. The cell is covered when the combination of technology, people, and process in that cell is functional. A SIEM with no analyst coverage is a logging system. It belongs in Networks/Identify (we can review logs after the fact) more than in Networks/Detect (we are detecting events as they occur).

This distinction is uncomfortable to apply rigorously because it reduces scores. A program that honestly assesses capability rather than tool presence will score lower than one that counts every purchased tool as a satisfied control. Lower scores are more accurate. They’re also more useful for prioritization, which is the point.

The fidelity question also applies to process controls. “We have an incident response plan” as a satisfied control is worth much less than “we have an incident response plan that was tabletop-tested in the last 12 months and updated based on the findings.” The CDM doesn’t prescribe how you define fidelity, but it gives you the cell structure to apply consistent fidelity standards across the assessment.

Tracking over time

A one-time CDM scorecard is a snapshot. The real value comes from running it periodically and tracking movement.

Quarterly or semi-annual re-scoring of the cells that showed red in the last assessment tells you whether remediation efforts are actually moving the needle. It also tells you whether cells that were green are staying green: security posture degrades without maintenance, and cells that score well in an initial assessment can slip as systems change, people leave, and processes atrophy.

The heat map over time also provides a narrative for leadership reporting that a flat compliance percentage can’t. “Our Networks/Detect cell moved from 45% to 71% this quarter because we completed SIEM tuning and hired two additional analysts” is a specific, credible progress report. “Our overall compliance score improved from 63% to 67%” is not.

For organizations preparing for regulatory examinations or external audits, the CDM scorecard approach also has a practical advantage: it produces evidence of systematic coverage thinking rather than point-in-time control attestation. Regulators are increasingly interested in whether organizations understand their own security posture, not just whether they’ve checked the required boxes.

Choosing your baseline

The CDM scorecard works with any checklist or control framework. The choice of which framework to use as the source of questions depends on your regulatory context, your industry, and what your peers are measuring against.

For most Japanese financial institutions: start with FISC. It’s the de facto standard and the questions have regulatory standing. Layer CIS Controls on top for the operational specificity that FISC sometimes lacks.

For organizations with international operations or cross-border data flows: NIST CSF as the organizing structure, with SP 800-53 or ISO 27001 Annex A for control specifics, depending on whether you’re US-regulation-aligned or international-standards-aligned.

For cloud-heavy environments: CIS Benchmarks for cloud platforms (AWS, Azure, GCP) map well to CDM cells, particularly in the Devices and Networks rows where cloud infrastructure sits. The shared responsibility model changes which cells you own versus which the cloud provider owns, which is worth making explicit in the scoring.

The framework you choose matters less than using it consistently across cells and over time. The value of the CDM scorecard approach is comparative: cell against cell, period against period. That value depends on consistent measurement methodology, not on which baseline checklist you start from.

Next: Article 7 — Maximum Possible vs. Minimum Required: Designing for Real Users