Cryptographic Manifest Signing

A backup manifest is the ledger the whole recovery process trusts: it names every artifact, records the digest each artifact must reproduce, and fixes the moment in time the set was captured. Within Automated Backup Integrity Check Implementation, the checksum validation pipeline proves that an artifact still matches the digest written in that manifest — but nothing so far proves the manifest itself was not rewritten. An attacker who can edit the manifest can substitute a corrupted artifact together with its matching digest and the pipeline will happily certify the swap as valid. Cryptographic manifest signing closes that gap by binding the manifest to a private key that lives outside the backup system, so a verifier can prove both that the manifest is intact and that it was produced by the authority that holds the key.

This control turns “the digests matched” into a claim you can defend under audit. A detached signature over the canonical manifest makes tampering evident (any edit invalidates the signature) and non-repudiable (only the key holder could have produced it), which is exactly the assurance a restore drill needs before it trusts a manifest enough to provision a sandbox. The signing key sits behind the security boundaries for DR environments, its use is classified through the same error categorization framework as any other integrity fault, and every signature and verification event is written to immutable audit trails so the chain of custody survives a post-incident review.

Architecture and Execution Workflow

Figure. Digests are assembled, the manifest is canonicalized and signed once at backup time, then every reader re-verifies the detached signature before a restore drill is allowed to trust the manifest.

The asymmetry is the point of the design: signing happens exactly once, on a trusted host that holds the private key, while verification happens many times — on every drill runner, in every region, months later — using only the public key. That separation means the material capable of forging a manifest never leaves the signing host, yet any number of untrusted verifiers can still establish that a manifest is authentic before acting on it.

Manifest Contents and Structure

A manifest is worth signing only if it captures everything a verifier needs to reject a substituted or stale backup. At minimum it carries a per-artifact digest and byte size, the backup capture time as an epoch, and a chain reference to the previous manifest so a gap in the sequence is detectable. The size field is not redundant with the digest: a truncated artifact that happens to collide on a weak comparison is still caught by a size mismatch, and size is far cheaper to check first.

import hashlib
from pathlib import Path
from typing import Dict


def build_manifest(root: Path, artifacts: Dict[str, Path],
                   backup_epoch: int, prev_manifest_id: str) -> dict:
    """Assemble a manifest of per-artifact sha256 digests, sizes and chain refs."""
    entries = {}
    for name, path in sorted(artifacts.items()):
        h = hashlib.sha256()
        size = 0
        with path.open("rb") as fh:
            for block in iter(lambda: fh.read(1024 * 1024), b""):
                h.update(block)
                size += len(block)
        entries[name] = {"sha256": h.hexdigest(), "size": size}
    return {
        "schema": "bv-manifest/1",
        "backup_epoch": backup_epoch,
        "prev_manifest_id": prev_manifest_id,
        "artifacts": entries,
    }

The chain reference (prev_manifest_id) is what makes a series of manifests tamper-evident rather than each one in isolation. If an attacker deletes a manifest to hide a bad backup generation, the next manifest’s back-pointer no longer resolves, and the verifier can flag the discontinuity. This is the same append-only discipline that immutable audit trails apply to telemetry, applied here to the integrity ledger itself.

Canonicalization Before Signing

A signature is computed over exact bytes, so the manifest must serialize to one and only one byte sequence regardless of who produced it or in what language. JSON is not canonical by default: key order, whitespace, and Unicode escaping all vary between encoders, and any of those differences flips the signature to invalid even when the semantic content is identical. Canonicalization pins those degrees of freedom before a single byte is signed.

import json


def canonicalize(manifest: dict) -> bytes:
    """Deterministic bytes for signing: sorted keys, no spaces, UTF-8, LF-free."""
    return json.dumps(
        manifest,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    ).encode("utf-8")

Sorting keys removes insertion-order ambiguity, the tight separators remove incidental whitespace, and fixing the encoding to UTF-8 removes escaping drift. The signer and every verifier must call the identical function; a mismatch here manifests as a phantom “invalid signature” on a manifest that was never actually tampered with, which is one of the most common and most confusing failures in a signing deployment.

Signing With an Asymmetric Key

The choice of primitive determines who can forge a manifest. A symmetric HMAC is fast and simple, but the verifier must hold the same secret the signer used, so every drill runner that can verify a manifest can also mint one — unacceptable when runners are numerous and less trusted than the backup control plane. An asymmetric signature breaks that symmetry: the signer holds a private key, verifiers hold only the public key, and possession of the public key confers no ability to sign. Ed25519 is the pragmatic default (small keys, fast, misuse-resistant); RSA-PSS is the fallback where a hardware module or compliance regime mandates RSA.

from cryptography.hazmat.primitives.asymmetric.ed25519 import (
    Ed25519PrivateKey,
    Ed25519PublicKey,
)


def sign_manifest(private_key: Ed25519PrivateKey, manifest: dict) -> bytes:
    """Produce a detached Ed25519 signature over the canonical manifest bytes."""
    return private_key.sign(canonicalize(manifest))


def verify_manifest(public_key: Ed25519PublicKey, manifest: dict,
                    signature: bytes) -> bool:
    """Return True only if the signature matches the canonical manifest."""
    from cryptography.exceptions import InvalidSignature
    try:
        public_key.verify(signature, canonicalize(manifest))
        return True
    except InvalidSignature:
        return False

The signature is detached: it is stored as a separate .sig file rather than embedded in the manifest, so the manifest bytes that were signed are the exact bytes on disk with nothing to strip out before verifying. Where an RSA regime is required, the equivalent uses padding.PSS with an MGF1-SHA256 mask and a SHA-256 hash, following the parameters in the cryptography library’s asymmetric signing documentation; the interface — sign the canonical bytes, verify against them — is identical.

Ed25519’s determinism is an operational asset worth naming. Unlike an RSA-PSS signature, which draws random salt on every signing and therefore produces a different signature each time, an Ed25519 signature over identical bytes is byte-identical on every run. That property makes signatures diffable and reproducible: a re-sign of an unchanged manifest yields the same .sig, so a signature that unexpectedly changed is itself a signal that the underlying manifest changed. It also removes a whole class of failures rooted in weak randomness on the signing host, which matters when the signer is a constrained appliance rather than a full server. The trade-off is key-agility: an organization already standardized on RSA hardware, or bound by a policy that enumerates RSA key sizes, keeps RSA-PSS and accepts the non-determinism. Neither choice changes the manifest format or the verification contract — only the bytes in the .sig file and the key type the verifier loads differ.

Verifying Before a Restore Trusts the Manifest

Verification is a gate, not a log line. The drill runner must verify the manifest signature before it reads a single digest out of the manifest, because an unsigned or badly-signed manifest is untrusted input and its digests are meaningless. The ordering matters: signature first, then per-artifact digest checks, then provisioning.

import sys
from pathlib import Path
from cryptography.hazmat.primitives.serialization import load_pem_public_key


def gate_restore(manifest: dict, signature: bytes, pubkey_pem: bytes) -> int:
    """Exit 0 only if the manifest signature is valid; else 1 (untrusted)."""
    public_key = load_pem_public_key(pubkey_pem)
    if not isinstance(public_key, Ed25519PublicKey):
        print("public key is not Ed25519", file=sys.stderr)
        return 1
    if not verify_manifest(public_key, manifest, signature):
        print("manifest signature INVALID -> refusing restore", file=sys.stderr)
        return 1
    print(f"manifest signature valid; {len(manifest['artifacts'])} artifacts trusted")
    return 0

Only after this gate returns 0 does the runner proceed to re-hash each artifact against the now-trusted digests. This is why signing is a prerequisite for, not a replacement of, checksum validation: the signature establishes that the digest list is authentic, and the checksum pipeline establishes that the stored bytes still reproduce those authentic digests.

The gate must also fail closed on the absence of evidence, not only on invalid evidence. A manifest that arrives without a signature file, a signature whose key id is not in the runner’s published key set, or a public key that fails to load are all treated exactly like an invalid signature: the restore is refused. The dangerous default is the opposite one, where a missing .sig is quietly interpreted as “unsigned, therefore skip the check” — that turns the strongest control into an opt-out an attacker can trigger simply by deleting a file. Encoding “no signature” and “bad signature” as the same non-zero outcome removes that escape hatch. For large manifests the verification cost is dominated by the per-artifact re-hash, not the single signature check, so the signature gate is effectively free to run first and short-circuits the expensive digest loop entirely whenever the manifest is untrusted.

Integration with DR Drill Orchestration

Manifest signing sits directly in front of the restore path and behind the backup path, so it composes with the adjacent stages rather than replacing them. At backup time, the signing step runs after the checksum validation pipeline has produced the digests, sealing the finished manifest. At drill time, the verification gate runs before sandbox provisioning is allowed to consume the backup, so no compute is spent standing up an environment around a manifest that cannot be trusted. A failed signature is not a soft warning: it short-circuits the drill exactly as a failed digest comparison does, and it is escalated through the same incident path so a forged-manifest event and a bit-rot event land in the same queue with distinct classifications. The full end-to-end signing module, including key generation and rotation, is developed in signing backup manifests with Python cryptography.

Key Custody, Rotation, and Timestamping

The security of every downstream verification collapses to one question: who can reach the private key. An offline or hardware-held signing key is the whole point of the asymmetric design, and the operational practices around it determine whether the guarantee holds.

• Offline signing key. Keep the private key on an HSM or an air-gapped signer, never on a drill runner. Runners receive only the public key, which cannot forge anything.
• Key rotation. Sign each manifest with a named key id and publish the retired public keys, so a manifest signed last quarter still verifies against the key that was current then. Rotation must never invalidate historical manifests.
• Countersignature and timestamping. A trusted timestamp over the signature proves the manifest existed at a claimed time, which defeats back-dating; a second countersignature from an independent custodian raises the bar from one compromised key to two.
• Revocation. Maintain a revocation list of key ids; a manifest whose signing key was later revoked is quarantined for review rather than silently trusted.

Error Classification and Thresholds

A signing verdict is binary, but the reasons a verification fails are not equally severe, and the orchestrator must route them differently.

Separating a bad signature (possible tampering) from an expired-but-valid key (a hygiene lapse) keeps the alerting proportionate: the first is an incident, the second is a ticket.

Telemetry and Compliance Output

Every sign and verify event emits structured telemetry so the signing control is observable and auditable rather than a silent library call.

Each verification writes a record — manifest id, signing key id, result, verifier host, and epoch — to write-once storage, so the chain of custody is reconstructable during an audit. This evidence maps directly onto the integrity and non-repudiation controls of frameworks such as NIST SP 800-53 and ISO 22301, and it is the raw material that immutable audit trails preserve.

Operational Best Practices

• Sign the canonical bytes, never the pretty-printed file. Always route both signer and verifier through one canonicalization function so formatting never masquerades as tampering.
• Verify before you read. Treat an unverified manifest as untrusted input; do not extract a digest, size, or path from it until the signature has passed.
• Hold the private key offline. Runners get the public key only; the signing key lives on an HSM or air-gapped host.
• Rotate with named key ids and publish retired keys. Historical manifests must keep verifying after a rotation.
• Timestamp and, for high-value data, countersign. A trusted timestamp defeats back-dating and a second custodian’s signature defeats single-key compromise.
• Fail closed. A missing signature file, an unknown key id, or a revoked key all refuse the restore rather than defaulting to trust.

Frequently Asked Questions

Related

• Signing backup manifests with Python cryptography — the full runnable module, including key generation and rotation.
• Checksum validation pipelines — produces the digests a manifest seals, and re-checks them once the signature is trusted.
• Immutable audit trails — where signing and verification events are preserved for forensic review.
• Security boundaries for DR environments — the trust zones that keep the signing key off the drill runners.
• Error categorization frameworks — routing a bad signature and an expired key as distinct events.

This topic is one component of the broader Automated Backup Integrity Check Implementation framework.