A Validation-Model Decision Matrix in Python

This page turns the decision framework described in the parent guide on validation-model selection into a single runnable tool that an orchestrator can call before it spends any compute on a drill. Applying the same verification depth to every artifact is the common failure: a tier-one payments ledger and a rebuildable analytics cache do not warrant the same rigor, yet a flat policy either over-spends on the cache or under-verifies the ledger. The tool below reads an artifact profile, scores it against a matrix encoded as data rather than nested conditionals, and returns one of three escalating models — a cheap manifest re-hash rooted in the checksum validation pipeline, a structural walk that borrows from page corruption scanning, or a full sandbox restore. Its scoring weights the recovery envelope from your RTO/RPO mapping directly, and its runtime estimate reuses the throughput assumptions that govern async batching for large datasets, so the model it names is one that can actually finish inside the window it was given.

Architecture and Execution Model

Five-stage validation-model decision flow A vertical flow: load the artifact profile, score criticality and the RTO/RPO regime, apply the data-encoded decision matrix, select one of three escalating validation models, then emit the model and an orchestrator exit code. Load artifact profile Score criticality + RTO/RPO Apply decision matrix Select validation model Emit model + exit code

Figure. The tool loads a profile, scores it, applies the data-encoded matrix, selects a model, and emits the model together with an exit code the orchestrator branches on.

The flow is deliberately linear and side-effect free: nothing is provisioned, nothing is hashed. The tool is a pure function from an artifact profile to a policy decision, which is exactly what makes it cheap to run on every artifact in a backup generation and safe to replay when an auditor asks why a given backup was verified the way it was.

Prerequisites

  • Python 3.8+ — the tool is standard-library only (dataclasses, enum, json, sys); no third-party packages are required.

    bash
    python3 --version   # 3.8 or newer
    
  • An artifact profile emitted upstream as JSON, carrying criticality (tier1/tier2/tier3), rto_minutes, rpo_minutes, size_gb, and blast_radius (the count of downstream systems that consume the restored data).

  • A calling orchestrator that treats the process exit code as a gate: 0 proceed, 1 escalate for a human decision, 2 abort on a malformed profile.

Production Implementation

The design goal is auditability. Rather than a tree of if criticality == ... and rto < ... branches, the policy lives in three data tables: a weight table that converts a profile into an integer criticality score, an ordered band table that maps a score to a model, and a throughput table that estimates whether the chosen model fits the recovery-time budget. Changing policy means editing a table, and every decision emits the exact table row that fired.

python
#!/usr/bin/env python3
"""Select an escalating backup-validation model from an artifact profile.

The tool scores a JSON artifact profile against a data-encoded decision matrix
and prints the mandated validation model plus a machine-readable justification
that a DR drill orchestrator can dispatch on.

Validation models, cheapest to most expensive:
    manifest-checksum   re-hash artifacts against a signed manifest
    structural-scan     open the artifact and walk its internal page structure
    full-restore        restore into a sandbox and run smoke tests

Exit codes (consumed by the orchestrator):
    0  a model was selected and fits the RTO budget -> proceed with the drill
    1  the mandated model cannot complete within RTO -> escalate for a decision
    2  usage / configuration error -> abort
"""
import json
import sys
from dataclasses import dataclass, field
from enum import IntEnum
from typing import Dict, List, Tuple


class Model(IntEnum):
    """Escalation order is the enum value; lower is cheaper."""

    MANIFEST_CHECKSUM = 1
    STRUCTURAL_SCAN = 2
    FULL_RESTORE = 3

    @property
    def label(self) -> str:
        return self.name.lower().replace("_", "-")


# --- Policy encoded as data, not control flow ---------------------------------

TIER_SCORE: Dict[str, int] = {"tier1": 50, "tier2": 30, "tier3": 10}

# (inclusive upper bound in minutes, points) evaluated in order, first match wins
RTO_BANDS: Tuple[Tuple[int, int], ...] = ((15, 25), (60, 15), (240, 8))
RPO_BANDS: Tuple[Tuple[int, int], ...] = ((5, 15), (60, 8), (1440, 3))

BLAST_POINTS_PER_SYSTEM = 3
BLAST_CAP = 21  # never let a wide fan-out alone dominate the tier signal

# Ordered high-to-low: the first row whose min_score <= score fires.
@dataclass(frozen=True)
class Band:
    rule_id: str
    min_score: int
    model: Model
    rationale: str


SCORE_BANDS: Tuple[Band, ...] = (
    Band("R1", 80, Model.FULL_RESTORE,
         "high criticality and a tight recovery envelope justify a live restore"),
    Band("R2", 45, Model.STRUCTURAL_SCAN,
         "moderate criticality warrants opening the artifact, not just hashing it"),
    Band("R3", 0, Model.MANIFEST_CHECKSUM,
         "low criticality is adequately covered by a signed-manifest re-hash"),
)

# GB per minute each model sustains on the validation fleet (capacity estimate).
THROUGHPUT_GB_PER_MIN: Dict[Model, float] = {
    Model.MANIFEST_CHECKSUM: 40.0,
    Model.STRUCTURAL_SCAN: 8.0,
    Model.FULL_RESTORE: 1.5,
}


@dataclass
class Profile:
    criticality: str
    rto_minutes: int
    rpo_minutes: int
    size_gb: float
    blast_radius: int

    @classmethod
    def from_dict(cls, raw: dict) -> "Profile":
        try:
            prof = cls(
                criticality=str(raw["criticality"]).lower(),
                rto_minutes=int(raw["rto_minutes"]),
                rpo_minutes=int(raw["rpo_minutes"]),
                size_gb=float(raw["size_gb"]),
                blast_radius=int(raw["blast_radius"]),
            )
        except (KeyError, TypeError, ValueError) as exc:
            raise ValueError(f"invalid profile: {exc}") from exc
        if prof.criticality not in TIER_SCORE:
            raise ValueError(f"unknown criticality tier {prof.criticality!r}")
        if prof.size_gb < 0 or prof.rto_minutes <= 0:
            raise ValueError("size_gb must be >= 0 and rto_minutes > 0")
        return prof


@dataclass
class Decision:
    model: Model
    score: int
    rule_id: str
    estimated_minutes: float
    fits_rto: bool
    reasons: List[str] = field(default_factory=list)


def _band_points(value: int, bands: Tuple[Tuple[int, int], ...]) -> int:
    for upper, points in bands:
        if value <= upper:
            return points
    return 0


def score_profile(prof: Profile) -> Tuple[int, List[str]]:
    """Convert a profile into an integer criticality score with a paper trail."""
    reasons: List[str] = []
    tier_pts = TIER_SCORE[prof.criticality]
    reasons.append(f"tier {prof.criticality} -> +{tier_pts}")

    rto_pts = _band_points(prof.rto_minutes, RTO_BANDS)
    reasons.append(f"RTO {prof.rto_minutes}m -> +{rto_pts}")

    rpo_pts = _band_points(prof.rpo_minutes, RPO_BANDS)
    reasons.append(f"RPO {prof.rpo_minutes}m -> +{rpo_pts}")

    blast_pts = min(prof.blast_radius * BLAST_POINTS_PER_SYSTEM, BLAST_CAP)
    reasons.append(f"blast radius {prof.blast_radius} -> +{blast_pts}")

    return tier_pts + rto_pts + rpo_pts + blast_pts, reasons


def select_band(score: int) -> Band:
    for band in SCORE_BANDS:
        if score >= band.min_score:
            return band
    return SCORE_BANDS[-1]  # R3 has min_score 0 and is the guaranteed fallback


def estimate_minutes(model: Model, size_gb: float) -> float:
    return size_gb / THROUGHPUT_GB_PER_MIN[model]


def decide(prof: Profile) -> Decision:
    """Score, select, then de-escalate non-critical artifacts that miss the RTO."""
    score, reasons = score_profile(prof)
    band = select_band(score)
    model = band.model
    reasons.append(f"score {score} matched {band.rule_id}: {band.rationale}")

    # Size makes an expensive model infeasible; de-escalate all but tier1, which
    # must keep its mandated depth and be escalated to a human instead.
    while True:
        minutes = estimate_minutes(model, prof.size_gb)
        if minutes <= prof.rto_minutes:
            return Decision(model, score, band.rule_id, round(minutes, 2), True, reasons)
        if prof.criticality != "tier1" and model > Model.MANIFEST_CHECKSUM:
            cheaper = Model(model - 1)
            reasons.append(
                f"{model.label} est {minutes:.1f}m exceeds RTO {prof.rto_minutes}m "
                f"-> de-escalate to {cheaper.label}")
            model = cheaper
            continue
        reasons.append(
            f"{model.label} est {minutes:.1f}m exceeds RTO {prof.rto_minutes}m "
            f"and cannot be de-escalated -> escalate")
        return Decision(model, score, band.rule_id, round(minutes, 2), False, reasons)


def main(argv: List[str]) -> int:
    if len(argv) != 2:
        print("usage: select_validation_model.py <profile.json>", file=sys.stderr)
        return 2
    try:
        raw = json.loads(open(argv[1], "r", encoding="utf-8").read())
        prof = Profile.from_dict(raw)
    except (OSError, json.JSONDecodeError, ValueError) as exc:
        print(f"config error: {exc}", file=sys.stderr)
        return 2

    decision = decide(prof)
    json.dump(
        {
            "model": decision.model.label,
            "score": decision.score,
            "rule_id": decision.rule_id,
            "estimated_minutes": decision.estimated_minutes,
            "fits_rto": decision.fits_rto,
            "justification": decision.reasons,
        },
        sys.stdout,
        indent=2,
    )
    sys.stdout.write("\n")
    return 0 if decision.fits_rto else 1


if __name__ == "__main__":
    sys.exit(main(sys.argv))

The decide function is where size and blast radius earn their place. Blast radius feeds the score, so a small but widely consumed dataset can be pulled up into a deeper model. Size feeds the feasibility loop: a model that the score mandated but that cannot finish inside the RTO is de-escalated one level at a time for anything below tier1, and each de-escalation is appended to the justification. A tier1 artifact is never silently downgraded — if its mandated model overruns the budget, the tool exits 1 and hands the conflict to an operator, because quietly verifying the most critical data less thoroughly is precisely the outcome the matrix exists to prevent.

Step-by-Step Execution Walkthrough

  1. Write a profile. Save the artifact’s attributes as JSON, for example {"criticality": "tier1", "rto_minutes": 30, "rpo_minutes": 5, "size_gb": 12, "blast_radius": 6}.
  2. Run the selector. python3 select_validation_model.py profile.json. The tool prints a JSON decision to stdout and diagnostics to stderr.
  3. Read the model. The model field is the string the orchestrator dispatches on: manifest-checksum, structural-scan, or full-restore.
  4. Read the exit code. echo $? returns 0 when the model fits the RTO, 1 when a mandated model overran the budget and needs a decision, 2 when the profile was malformed.
  5. Archive the justification. Persist the justification array with the drill record; it is the auditable trail of every table row that contributed to the verdict.

Verification and Expected Output

A tier-one, tightly bounded, small artifact scores high and is affordable to fully restore, so the tool selects full-restore and exits 0:

text
$ python3 select_validation_model.py profile.json
{
  "model": "full-restore",
  "score": 98,
  "rule_id": "R1",
  "estimated_minutes": 8.0,
  "fits_rto": true,
  "justification": [
    "tier tier1 -> +50",
    "RTO 30m -> +15",
    "RPO 5m -> +15",
    "blast radius 6 -> +18",
    "score 98 matched R1: high criticality and a tight recovery envelope justify a live restore"
  ]
}
$ echo $?
0

Success means the printed model and the exit code agree: a 0 exit always carries "fits_rto": true, and the orchestrator can launch that model without re-checking the budget. A 1 exit still prints a model — the mandated one — but with "fits_rto": false, signalling that the drill must not auto-start until an operator either widens the RTO or accepts a shallower model.

Failure Modes and Troubleshooting

Symptom Cause Remediation
Exit 2, stderr unknown criticality tier Profile criticality is not one of tier1/tier2/tier3 Normalize the upstream tier vocabulary to the three canonical tiers before emitting the profile
Exit 2, stderr invalid profile A required key is missing or a numeric field is non-numeric Validate the profile schema at emit time; every key in Profile.from_dict is mandatory
Exit 1 on a tier1 artifact Mandated full-restore overruns the RTO for the artifact size Widen the RTO, pre-stage a warm sandbox, or split the artifact so a restore fits the window
A large tier3 dataset selects manifest-checksum unexpectedly Feasibility loop de-escalated the scored model to fit the RTO Expected behavior; inspect the justification for the de-escalation lines, and raise throughput if deeper checks are required
Score seems too low for a critical dataset blast_radius was reported as 0 or the RTO/RPO were loose Confirm the upstream census of downstream consumers; the recovery envelope is a first-class input, not a hint
All artifacts resolve to full-restore Throughput table over-estimates fleet capacity, so nothing ever de-escalates Calibrate THROUGHPUT_GB_PER_MIN against measured restore rates on your validation fleet

Integration Notes

Because the tool is a pure exit-code gate, it slots in front of any scheduler with no adapter. In an Airflow DAG, a PythonOperator (or a BashOperator invoking the script) runs first; on exit 0 it branches to the task implementing the returned model, on exit 1 it routes to a human-approval task, and on exit 2 it fails the run outright. Under Celery, dispatch the selection as a task whose return value is the model string and whose non-zero mapping raises, so an event-driven drill triggered by a fresh backup picks its depth in milliseconds. Feed the emitted justification and rule_id into your error categorization frameworks so a policy escalation (exit 1) is recorded distinctly from a malformed profile (exit 2), and treat the whole decision as the opening record of the drill that the broader validation-model selection workflow governs.

Frequently Asked Questions

Why encode the matrix as data tables instead of if/else branches?

A branch tree hides policy inside control flow, so an auditor cannot see which rule fired without reading the code path. Tables make the policy inspectable: the tool emits the exact rule_id and the point contributions that produced the score, and changing a threshold is a one-line data edit rather than a refactor that risks reordering conditions.

Why is a tier1 artifact never silently de-escalated?

De-escalation exists to keep a validation model inside the RTO budget, but for the most critical data, shallow verification is a worse outcome than a missed window. When a tier1 artifact's mandated model overruns the budget the tool exits 1 and surfaces the conflict, so an operator decides whether to widen the RTO, pre-warm a sandbox, or accept the risk explicitly.

How does blast radius change the selected model?

Blast radius is the count of downstream systems that consume the restored data, and it adds to the criticality score up to a cap. A physically small dataset that many services depend on therefore scores higher than its size alone would suggest, which can pull it from a manifest re-hash up into a structural scan or full restore.

Where do the throughput numbers come from?

The THROUGHPUT_GB_PER_MIN table is a capacity estimate for your validation fleet and must be calibrated against measured rates. If it over-states capacity, expensive models look feasible when they are not; if it under-states, the tool de-escalates too aggressively. Re-measure after any fleet or storage-tier change.

This tool is one component of the broader validation-model selection guide, and reads the recovery envelope defined by the NIST SP 800-34 contingency planning guide through the lens of Python’s dataclasses module.