Design YouTube: Scaling Bottlenecks

Once a candidate has laid out a solid baseline design for a YouTube-style platform, experienced interviewers pivot to scaling follow-ups. This is where the interview gets interesting, and where most candidates who memorized a static diagram start to struggle. This article covers the four bottlenecks that come up most often.

Bottleneck 1: Storage Growth

YouTube-scale platforms ingest an enormous, ever-growing volume of video. At 500 hours uploaded per minute, storage doesn’t grow linearly in a comfortable way — it compounds, and multiplying by multiple transcoded resolutions per video makes it worse (a single upload might become 5-6 stored variants).

The naive approach — store every resolution of every video forever on high-performance storage — becomes prohibitively expensive.

The production answer is storage tiering:

• Hot tier (SSD-backed, fast access): recently uploaded and frequently-viewed videos.
• Warm tier: moderately popular content, cheaper storage class, slightly higher retrieval latency acceptable.
• Cold/archive tier: rarely-viewed long-tail content, using object storage archive classes (like S3 Glacier-equivalent) at a fraction of the cost, with retrieval latency measured in minutes rather than milliseconds.

A background job monitors access patterns and automatically migrates videos between tiers based on recent view frequency — this is the same “access-frequency-driven tiering” pattern used across large-scale storage systems generally, and citing it as a known pattern (not something you’re inventing on the spot) scores well.

Additionally, lazy transcoding (mentioned in the architecture article) directly addresses storage growth: don’t generate and store every resolution upfront for every video — generate on first request for lower-priority resolutions, cache the result, and let unused variants never get created at all for the long tail of low-view videos.

Bottleneck 2: Hot Video Caching

When a video goes viral, request volume for that single piece of content can spike by orders of magnitude within minutes. This creates a very different problem from steady-state traffic: a small number of keys (video IDs) receiving a disproportionate share of all requests — the classic “hot key” problem.

Why naive caching isn’t enough: even with a CDN, if a single edge node’s cache entry expires and thousands of requests arrive in the same instant, they can all miss simultaneously and stampede the origin (see thundering herd below).

Mitigations to discuss:

• Extended TTLs for trending content. Detect rising view velocity and proactively extend cache TTL / pre-warm additional edge nodes before the spike fully materializes.
• Multi-region replication for hot content. Rather than relying on lazy cache population, proactively push a viral video’s segments to edge nodes in regions showing early traffic growth.
• Request coalescing at the edge. If multiple requests for the same uncached segment arrive close together, the edge node should only forward one request to origin and fan the response out to all waiting clients, rather than sending N duplicate requests to origin.

Bottleneck 3: Thundering Herd Problem

This is a specific, well-known failure pattern worth naming explicitly in an interview: when a cached resource expires and a large number of concurrent requests all miss the cache at once, they all hit the origin/database simultaneously, potentially overwhelming it — sometimes badly enough to cause a cascading outage.

For a video platform, this shows up in a few places:

• CDN cache expiry on a wildly popular video, as described above.
• Metadata cache expiry for a viral video’s info (title, view count, channel data) — if the read-through cache entry expires during peak traffic, every request floods the metadata database simultaneously.

Standard mitigations:

• Request coalescing / single-flight, so only one request per cache key actually reaches the backend while others wait on that in-flight result.
• Staggered/jittered TTL expiration, so cache entries for popular content don’t all expire at exactly the same moment.
• Stale-while-revalidate caching, serving the slightly-stale cached value immediately while asynchronously refreshing it in the background, rather than blocking new requests on a fresh fetch.

Naming “thundering herd” explicitly and describing at least two of these mitigations is one of the highest-signal moments in a scaling-focused interview round.

Bottleneck 4: Live Streaming at Scale

Live streaming is architecturally distinct from video-on-demand, and interviewers frequently use it as a capstone follow-up to see if a candidate can adapt their design rather than just reciting a memorized diagram.

Why VOD architecture doesn’t directly apply:

• There’s no “upload then transcode then serve” pipeline — video must be ingested, transcoded, and distributed in near real time, with latency budgets measured in seconds, not the minutes acceptable for VOD processing.
• Chunked transcoding (splitting a full file and parallelizing across workers) doesn’t work the same way, since the “file” doesn’t exist yet — it’s an ongoing stream.

Core live streaming architecture:

• The broadcaster’s client encodes and pushes a continuous stream to an ingest server using a low-latency streaming protocol.
• A real-time transcoding pipeline processes the incoming stream in small segments (a few seconds each) as they arrive, producing multiple bitrate variants continuously rather than as a single batch job.
• Each transcoded segment is immediately pushed to CDN edge nodes and made available in the stream’s manifest, which viewers’ players poll/refresh to discover new segments as they become available.
• Viewers experience some inherent latency (typically 5-30 seconds behind true real-time, depending on how aggressively low-latency protocols are used) due to the encode-transcode-distribute pipeline, which is an acceptable and expected tradeoff to mention explicitly.

Comparison: VOD vs. Live Streaming Architecture

Putting It Together in an Interview

When an interviewer escalates to scaling questions, the strongest signal you can give is naming the bottleneck precisely (storage growth, hot-key caching, thundering herd, or live-streaming’s fundamentally different pipeline) before proposing a fix. Candidates who jump straight to “we’d add more servers” without diagnosing the specific failure mode tend to score lower, even if the underlying instinct (scale horizontally) isn’t wrong.

A good closing move, time permitting, is to explicitly rank these bottlenecks by likely business impact: a thundering-herd outage during a viral event is a five-alarm production incident, while gradual storage cost growth is a slower-moving optimization problem — showing you can prioritize, not just enumerate problems, is what senior interviewers are listening for.

Further Reading

These scaling patterns — hot-key mitigation, thundering herd, tiered storage — reappear across almost every system design prompt, not just video platforms. The 0-to-1 SWE Interview Playbook (Amazon: https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20) collects these recurring patterns into a single reusable toolkit, so you can recognize and apply them regardless of which specific system an interviewer asks you to design.

If you haven’t already, read our companion articles on the baseline YouTube system design interview approach and the detailed component architecture to build the full picture before tackling these harder follow-ups.