1. Core Requirements & Scale Estimations
Twitter (X) processes ~500 million tweets per day (~6,000 tweets/sec average, ~25,000 peak) and serves ~300,000 timeline reads per second. The read-to-write ratio is heavily read-dominant (~50:1).
The primary performance metric is Home Timeline latency: delivering the most recent tweets from followed accounts in under 100 milliseconds.
2. Fan-Out on Write vs Fan-Out on Read
Fan-out on Write (Push): When User A posts a tweet, the Fan-Out Service looks up all followers and inserts the Tweet ID directly into each follower's in-memory Redis timeline list. Reads are blazing fast (O(1) Redis LRANGE). However, when a celebrity with 80M followers tweets, one write triggers 80 million Redis updates, creating severe backpressure.
Fan-Out on Read (Pull): Tweets are only stored in the author's tweet list. When a follower opens their app, the system queries tweets from all followed users and performs a k-way merge. This saves write capacity but makes timeline loads unacceptably slow.
3. The Hybrid Fan-Out Architecture
Modern timeline engines use a hybrid approach: standard users (<50k followers) use Fan-Out on Write. For verified celebrity accounts, tweets are excluded from fan-out workers.
When an active user loads their timeline, the system fetches their pre-computed Redis timeline and dynamically blends in recent tweets from any celebrities they follow at query time.
4. Timeline Storage & Snowflake ID Generation
Redis timeline lists only store 64-bit Tweet IDs, capped at the most recent 800 items per user. Full tweet payloads (text, media URLs, author info) are hydrated in batch via Memcached.
Tweets are ordered chronologically using Twitter Snowflake IDs: 64-bit unique identifiers composed of timestamp (41 bits), datacenter/worker ID (10 bits), and sequence number (12 bits).
Model this architecture on the HLDForge Canvas
Turn static box diagrams into living, collaborative systems. Drag and drop microservices, Kafka clusters, and cache tiers. Run automated SPOF detection and STRIDE threat modeling in real time.
Written by HLDForge Architecture Team
Staff Distributed Systems Architect
The HLDForge Architecture Team publishes practical engineering guides, real-world case studies, and high-concurrency blueprints to help engineers build scalable and fault-tolerant software.