1. RPO and RTO Fundamentals
Recovery Point Objective (RPO): The maximum acceptable amount of data loss measured in time (e.g. 'We can tolerate losing at most 5 minutes of data').
Recovery Time Objective (RTO): The maximum acceptable duration of service downtime before restoration (e.g. 'The system must be fully operational within 15 minutes of an outage').
2. Disaster Recovery Strategy Spectrum
1. Backup & Restore (Cold Standby): Cheapest, highest RTO/RPO (hours/days).
2. Pilot Light: Core databases replicate continuously to standby region; compute instances are turned off and spun up via Terraform during disasters (RTO: 10-30 mins, RPO: <1 min).
3. Warm Standby: Scaled-down compute fleet runs continuously in standby region, ready to scale up via autoscaling (RTO: minutes).
4. Active-Active Multi-Region: Full capacity serving traffic in both regions simultaneously (RTO: near zero, RPO: near zero, but highest operational cost).
3. Split-Brain Prevention & Fencing
During cross-region network partitions, both regions might assume the other has failed and promote their local databases to Primary. We prevent split-brain using an odd-numbered witness quorum (e.g. ZooKeeper/Consul nodes across 3 regions) and fencing tokens.
Model this architecture on the HLDForge Canvas
Turn static box diagrams into living, collaborative systems. Drag and drop microservices, Kafka clusters, and cache tiers. Run automated SPOF detection and STRIDE threat modeling in real time.
Written by HLDForge Architecture Team
Principal Cloud Infrastructure Architect
The HLDForge Architecture Team publishes practical engineering guides, real-world case studies, and high-concurrency blueprints to help engineers build scalable and fault-tolerant software.