Generating SOC 2 and ISO 22301 Evidence

This page builds one concrete deliverable from the broader compliance evidence and control mapping guide: a Python pipeline that turns raw DR-drill telemetry into a signed evidence bundle mapped to the SOC 2 Availability criteria (A1.2 backup and recovery, A1.3 recovery-plan testing) and ISO 22301 clauses 8.4 (business continuity procedures) and 8.5 (exercising and testing). It complements the NIST projection in mapping backup validation to NIST SP 800-34, reads the same outcome stream your checksum validation pipeline and restore drills emit, and — because a drill only counts as passing when it recovered inside the window your RTO and RPO mapping defines — carries each record’s recorded verdict forward untouched. The pipeline hashes every evidence record, rolls the records up per trust-service criterion and per ISO clause, and emits a single detached-signed bundle an auditor can verify offline, exiting non-zero when any mapped criterion has no passing evidence.

Architecture and Execution Model

Figure. Drill telemetry is normalized, mapped to the Availability criteria and continuity clauses, hashed per record, rolled up per criterion, and emitted as one signed bundle.

Two frameworks are served by one pass because their recovery-testing expectations rhyme. SOC 2’s Availability category asks, through A1.2, that the entity maintain backup and recovery infrastructure, and through A1.3 that recovery procedures be tested; ISO 22301 asks, through clause 8.4, that continuity procedures be documented and operable, and through clause 8.5 that they be exercised and evaluated. The same drill outcome therefore substantiates a SOC 2 criterion and an ISO clause at once, and fanning each telemetry event out to every criterion it supports — rather than forcing an auditor to cross-walk two separate reports — keeps the evidence internally consistent. A single restore drill that recovered inside its recovery-time envelope is one fact expressed against four coordinates, not four independent claims that could drift apart.

Prerequisites

• Python 3.9+ on the executor host.

• The cryptography package for per-record hashing and Ed25519 bundle signing:

  pip install cryptography
  

• An Ed25519 signing key held by the compliance function, kept out of the drill environment so evidence cannot be self-signed by the system under test.

• A drill telemetry file (JSON array) with one record per validation or restore run, each carrying component, artifact_uri, drill_id, verdict, and an ISO-8601 UTC completed_at.

Production Implementation

The mapping table declares which criterion each component’s evidence supports. A backup-integrity component feeds A1.2 and clause 8.4; a restore-drill component feeds A1.3 and clause 8.5. Normalization then projects each telemetry event into a canonical evidence record and hashes it, so any later mutation of the record is detectable independent of the bundle signature. The hash is computed over a canonical JSON encoding — keys sorted, whitespace stripped — because a record that hashes differently after a harmless re-serialization is worthless as tamper evidence. Sorting the keys makes the digest a function of the record’s meaning, not of the incidental order a dictionary happened to iterate in.

#!/usr/bin/env python3
"""Transform DR-drill telemetry into a signed SOC 2 / ISO 22301 evidence bundle.

Usage:
    python3 evidence_bundle.py <telemetry.json> <signing_key.pem> <out_bundle.json>

Exit codes:
    0  bundle emitted; every mapped criterion has passing evidence
    1  bundle emitted, but at least one mapped criterion is uncovered
    2  usage / configuration / key error
"""
from __future__ import annotations

import hashlib
import json
import sys
from dataclasses import dataclass, asdict

from cryptography.exceptions import UnsupportedAlgorithm
from cryptography.hazmat.primitives.serialization import load_pem_private_key
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey

# component -> criteria it supplies evidence for
CRITERION_MAP = {
    "checksum-validator":   ["SOC2-A1.2", "ISO-8.4"],
    "restore-drill-runner": ["SOC2-A1.3", "ISO-8.5"],
}
# every criterion we claim to cover; a claimed criterion with no pass is a gap
CLAIMED_CRITERIA = ["SOC2-A1.2", "SOC2-A1.3", "ISO-8.4", "ISO-8.5"]


@dataclass(frozen=True)
class EvidenceRecord:
    criterion: str
    component: str
    artifact: str
    drill_id: str
    timestamp: str
    verdict: str
    record_hash: str


def _hash(payload: dict) -> str:
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode("utf-8")).hexdigest()


def normalize(telemetry: list[dict]) -> list[EvidenceRecord]:
    """Fan each telemetry event out to one record per criterion it supports."""
    records: list[EvidenceRecord] = []
    for event in telemetry:
        criteria = CRITERION_MAP.get(event["component"], [])
        for criterion in criteria:
            core = {
                "criterion": criterion,
                "component": event["component"],
                "artifact": event["artifact_uri"],
                "drill_id": event["drill_id"],
                "timestamp": event["completed_at"],
                "verdict": event["verdict"],
            }
            records.append(EvidenceRecord(record_hash=_hash(core), **core))
    return records

The rollup groups records by criterion and marks a criterion covered when it holds at least one pass record. It reports the passing count alongside the total, so an auditor can see not just that a criterion was met but how many independent drills met it — a criterion covered by a single record is weaker evidence than one covered by a dozen, even though both clear the boolean bar. The bundle serializes the ordered records and the rollup, hashes that canonical payload, and signs the digest with the detached Ed25519 key. Signing the whole ordered payload — rather than each record individually — means reordering or dropping a record invalidates the signature just as surely as editing one. Ed25519 is chosen over RSA here for the compactness of its signatures and its resistance to the padding and key-size footguns that make RSA signing error-prone in a hand-rolled pipeline.

def rollup(records: list[EvidenceRecord]) -> dict:
    """Per-criterion coverage: covered iff a passing record exists."""
    summary: dict[str, dict] = {}
    for criterion in CLAIMED_CRITERIA:
        matching = [r for r in records if r.criterion == criterion]
        passing = [r for r in matching if r.verdict == "pass"]
        summary[criterion] = {
            "records": len(matching),
            "passing": len(passing),
            "covered": bool(passing),
        }
    return summary


def build_bundle(records: list[EvidenceRecord], summary: dict,
                 key: Ed25519PrivateKey) -> dict:
    body = {
        "schema": "dr-evidence-bundle/v1",
        "criteria": summary,
        "records": [asdict(r) for r in records],
    }
    canonical = json.dumps(body, sort_keys=True, separators=(",", ":"))
    signature = key.sign(canonical.encode("utf-8"))
    return {
        "body": body,
        "body_sha256": hashlib.sha256(canonical.encode("utf-8")).hexdigest(),
        "signature_hex": signature.hex(),
        "algorithm": "ed25519",
    }


def main() -> int:
    if len(sys.argv) != 4:
        print("usage: evidence_bundle.py <telemetry.json> <key.pem> <out.json>",
              file=sys.stderr)
        return 2
    try:
        telemetry = json.loads(open(sys.argv[1], "r", encoding="utf-8").read())
        key_bytes = open(sys.argv[2], "rb").read()
        key = load_pem_private_key(key_bytes, password=None)
    except (OSError, ValueError, json.JSONDecodeError) as exc:
        print(f"config error: {exc}", file=sys.stderr)
        return 2
    if not isinstance(key, Ed25519PrivateKey):
        print("signing key must be Ed25519", file=sys.stderr)
        return 2
    try:
        records = normalize(telemetry)
    except (KeyError, UnsupportedAlgorithm) as exc:
        print(f"malformed telemetry: {exc}", file=sys.stderr)
        return 2
    summary = rollup(records)
    bundle = build_bundle(records, summary, key)
    with open(sys.argv[3], "w", encoding="utf-8") as fh:
        json.dump(bundle, fh, indent=2)
    gaps = [c for c, s in summary.items() if not s["covered"]]
    for criterion, stats in summary.items():
        mark = "COVERED" if stats["covered"] else "GAP    "
        print(f"{mark} {criterion:<10} passing={stats['passing']}/{stats['records']}")
    if gaps:
        print(f"UNCOVERED CRITERIA: {', '.join(gaps)}", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

Step-by-Step Execution Walkthrough

1. Generate the signing key once. Have the compliance function create an Ed25519 private key in PEM form and publish only the matching public key to auditors; never store the private key in the drill environment.
2. Export drill telemetry. Collect the run outcomes into a JSON array where each element carries component, artifact_uri, drill_id, verdict, and completed_at.
3. Run the pipeline. python3 evidence_bundle.py telemetry.json signing_key.pem bundle.json. The per-criterion rollup prints to stdout and the signed bundle is written to bundle.json.
4. Read the exit code. echo $? — 0 means every claimed criterion has passing evidence, 1 means the bundle was still written but a criterion is uncovered, 2 signals a bad key, telemetry, or arguments.
5. Hand the bundle to the auditor. They verify signature_hex against body with the published public key, then confirm each record’s record_hash matches its content, establishing chain of custody without contacting the drill system.

Verification and Expected Output

A fully covered run prints the rollup and exits 0:

COVERED SOC2-A1.2  passing=3/3
COVERED SOC2-A1.3  passing=1/1
COVERED ISO-8.4    passing=3/3
COVERED ISO-8.5    passing=1/1

If no restore drill ran in the window, the two recovery-testing criteria fall out and the process exits 1 while still writing the bundle for the record:

COVERED SOC2-A1.2  passing=3/3
GAP     SOC2-A1.3  passing=0/0
COVERED ISO-8.4    passing=3/3
GAP     ISO-8.5    passing=0/0
UNCOVERED CRITERIA: SOC2-A1.3, ISO-8.5

Success means the telemetry parsed, the key loaded as Ed25519, every record hashed, and each claimed criterion held a passing record. An auditor re-verifying the bundle recomputes the SHA-256 over body, checks it equals body_sha256, and verifies signature_hex — any of those failing proves the bundle was altered after signing.

Failure Modes and Troubleshooting

Integration Notes

The pipeline exits with clean POSIX codes, so it slots into any scheduler as a gate. Run it as the terminal task of an Airflow evidence DAG so a 1 exit surfaces an uncovered criterion in the run history; dispatch it from a Celery task that raises on non-zero so a completed drill triggers immediate bundling; or invoke it from cron and archive each bundle.json into the immutable audit trails store. Route uncovered criteria back into drill scheduling and orchestration so the next cycle exercises exactly the recovery-testing controls that lapsed. For authoritative definitions, consult the AICPA Trust Services Criteria for the Availability criteria, the ISO 22301 standard for clauses 8.4 and 8.5, and the cryptography Ed25519 documentation for the signing primitives.

Frequently Asked Questions

Related

• Compliance evidence and control mapping — the parent guide defining the evidence model this bundle serializes.
• Mapping Backup Validation to NIST SP 800-34 — the sibling projection for the contingency-planning control family.
• Immutable audit trails — the write-once store each signed bundle is archived into.
• Validation telemetry and metrics — the upstream source of the drill records this pipeline normalizes.

This pipeline is one component of the broader compliance evidence and control mapping guide.