1. The Deceptive Simplicity of CAP
Eric Brewer's CAP Theorem states that a distributed data store can simultaneously provide at most two out of three guarantees: Consistency (linearizability), Availability (every non-failing node returns a non-error response), and Partition Tolerance (system continues operating despite arbitrary dropped or delayed messages).
In real-world networks (AWS, GCP, on-prem fibers), network partitions are an immutable physical reality. Fibers get cut, switches fail, and BGP routes flap. Therefore, Partition Tolerance (P) is mandatory. The real decision is between CP and AP when a partition occurs.
2. PACELC: What Happens During Normal Operation?
The primary limitation of CAP is that it only describes system behavior when a partition exists. Partitions are rare (accounting for <0.1% of operational time). What about the other 99.9% of the time?
Daniel Abadi formulated PACELC: If there is a Partition (P), how does the system trade off Availability (A) and Consistency (C); Else (E), how does it trade off Latency (L) and Consistency (C)?
- PC/EC (e.g. Google Spanner, CockroachDB): Consistent under partition; Consistent (higher latency) during normal operation.
- PA/EL (e.g. Amazon DynamoDB, Apache Cassandra): Available under partition; Latency-optimized (eventual consistency) during normal operation.
- PC/EL (e.g. MongoDB, MySQL Group Replication): Consistent under partition; Latency-optimized reading from replicas during normal operation.
3. Production Decision Matrix
Financial ledgers, inventory counts, and authentication systems demand PC/EC semantics. Social media timelines, viewing telemetry, and notification services thrive on PA/EL. Aligning business invariants with PACELC guarantees avoids costly architectural rewrites.
Model this architecture on the HLDForge Canvas
Turn static box diagrams into living, collaborative systems. Drag and drop microservices, Kafka clusters, and cache tiers. Run automated SPOF detection and STRIDE threat modeling in real time.
Written by HLDForge Architecture Team
Principal Distributed Systems Architect
The HLDForge Architecture Team publishes practical engineering guides, real-world case studies, and high-concurrency blueprints to help engineers build scalable and fault-tolerant software.