Before shipping it, separate protocol facts, product promises, and operating cost. Mixing those layers produces confident but incorrect decisions. A successful backup upload proves only a file was written, not that WAL, object versions, encryption keys, or runnable config are present. Restore tests the dependency graph.
Data design covers atomic writes, concurrent access, schema migration, crash recovery, and cleanup. One successful save says nothing about power loss, full storage, or reopening with an older version.
The parts that make the design practical
List non-negotiable invariants before selecting performance knobs. Tuning can roll out gradually; identity, permission, and terminal-state rules cannot drift at runtime.
- Manifests record schema, time range, checksums, and key versions; restore to a target time in isolation, run invariants and read-only smoke tests, then destroy safely.
- Bound every input by size, count, and time, returning a stable actionable error code when a budget is exceeded.
- Ship conservative defaults, server-side ceilings, and a rollout switch instead of trusting browser-provided numbers as resource budgets.
The delivery standard for A Backup Counts Only After a Restore Drill is a usable normal path, convergent failures, bounded resources, and a state users can understand. The result is a production capability that can be explained, degraded safely, and rolled back—not a demo that works once.
Keep false assumptions out of production
Boundaries turn hidden assumptions into incidents. Weak networks, refresh, concurrency, and capacity need combined coverage because retries can hide each one in isolation.
- Backups sharing production credentials can be deleted by one compromise, while untested old schemas may be unreadable by current code during crisis.
- A stale response arriving after a new task can overwrite healthy state or restart cancelled work without version fencing.
- Without backpressure or quota, a slow consumer raises memory, queue depth, and tail latency until unrelated users are affected.
What the release gate should inspect
Build golden cases from known inputs and controlled faults, then align production metrics with those results. Verification extends to production only when signals detect the same degradation early.
- Quarterly choose a random point and have a non-author restore it; measure actual RPO/RTO, missing dependencies, checksum failures, and manual steps.
- Run one hundred start, fail, retry, and cancel cycles; handles, listeners, queues, and temporary data must return to baseline.
- Allowlist log and analytics fields, proving payloads, secrets, full IP addresses, and identifying data never leave the device.
The release standard is practical: the normal path is fast, abnormal paths converge, recovery never overrides an explicit user decision, and operators can diagnose faults from limited, privacy-safe evidence.