1. Core Requirements & Scale Estimation
YouTube handles over 500 hours of video uploaded every minute and serves over 1 billion hours of video daily. When designing such a platform, we must separate the Write Path (video ingestion, transcoding, packaging) from the Read Path (metadata query, adaptive bitrate video delivery via CDNs).
Back-of-the-envelope calculations: 500 hours/min = ~8.3 hours of video per second. At an average raw bit rate of 50 Mbps, ingress bandwidth is ~415 Gbps. On the egress side, 1 billion hours/day translates to ~11.5 million concurrent video streams. Assuming an average streaming bit rate of 3 Mbps, peak egress bandwidth exceeds 34 Tbps.
- Daily Active Users (DAU): 2.5 Billion
- Ingress: 500 hours uploaded per minute (~30,000 hours/hr)
- Egress: >34 Tbps peak CDN bandwidth
- Storage: >150 Petabytes per year of encoded video assets
2. High-Level Architecture & Ingestion Topology
The architecture is partitioned into four major tiers: Client Edge (DNS Anycast & CloudFront/Akamai CDNs), Ingestion Cluster, Asynchronous Transcoding DAG Engine, and Metadata / Search Microservices.
Clients never upload large multi-gigabyte video files directly to application servers in a single HTTP POST request. Instead, we use chunked resumable multipart uploads directly to an S3/GCS Object Storage bucket via presigned URLs, coordinated by an Upload Gateway.
3. Transcoding DAG Pipeline & Adaptive Bitrate Streaming (ABR)
Raw uploaded videos must be converted into multiple resolutions (1080p, 720p, 480p, 360p) and codecs (H.264/AVC, H.265/HEVC, VP9, AV1). To deliver smooth playback on variable mobile connections, we split videos into 2- to 6-second segments packaged as HLS (HTTP Live Streaming) or MPEG-DASH manifests.
A DAG orchestration engine processes video in parallel: a video is split into 10-second GOP chunks, individual workers transcode chunks concurrently, and a final packager stitches the HLS master playlist (.m3u8).
4. Database Schema & Caching Architecture
Video metadata requires a read-heavy relational or distributed document store (like PostgreSQL with Vitess or Cassandra). Views, likes, and comments have different consistency requirements.
View counts are decoupled from synchronous database writes: viewing events are pushed to an Apache Kafka topic, aggregated in Apache Flink in 10-second tumbling windows, and flushed in batch to Redis and Cassandra to prevent database hotspots.
5. Single Points of Failure (SPOFs) & Failure Modes
A critical bottleneck in video streaming is the cache thundering herd: when a viral video is published, millions of users request the exact same HLS manifest simultaneously. We mitigate this using HTTP Request Collapsing in Edge Varnish/Nginx proxies and stale-while-revalidate caching headers.
For multi-region resilience, transcode outputs are replicated across 3 cloud regions, while CDN origins utilize origin shield layers to protect internal object stores from high egress spikes.
Model this architecture on the HLDForge Canvas
Turn static box diagrams into living, collaborative systems. Drag and drop microservices, Kafka clusters, and cache tiers. Run automated SPOF detection and STRIDE threat modeling in real time.
Written by HLDForge Architecture Team
Staff Distributed Systems Engineer
The HLDForge Architecture Team publishes practical engineering guides, real-world case studies, and high-concurrency blueprints to help engineers build scalable and fault-tolerant software.