Design YouTube

Alice records a 2 GB cooking video on her phone and taps Upload. The progress bar should talk to object storage, not to an API pod. An hour later the video is watchable at 1080p on Wi-Fi and 360p on a train. That night it goes viral. Fifty thousand people hit play in one minute. The view number on the watch page should move. The watch page itself must not die.

Those are different problems glued together by one product. Upload is a large, rare write. Playback is a huge, cacheable read of bytes we must not put on the application server. A view is a tiny event that would melt PostgreSQL if we UPDATE videos SET views = views + 1 on every play.

The main design question is:

How do we take a large file from a creator, turn it into many bitrates, serve it from a CDN, and count enormous view traffic, without making metadata, transcoding, or the database share one fate?

We will start with metadata in PostgreSQL and files that never enter the API process. Transcoding, counters, search, and the home feed appear when a simpler path fails.

1. Clarify the problem

“Design YouTube” includes ads, live streaming, Shorts ranking, Content ID, and a global CDN. That is too broad for one interview.

I would ask:

• Upload and playback only, or also comments, likes, subscriptions, search, and Home?
• Live streaming in scope?
• Must we implement real transcoding, or a pipeline with a pluggable encoder?
• How many DAU and uploads should we plan for?
• Recommendations: a real model, or candidate generation plus a simple ranker?
• Copyright matching and monetization?

If the interviewer gives no extra constraints, I would state:

Product              Upload, process, play, views, likes, comments,
                     subscriptions, watch history, search, Home
Not in scope         Live, ads, Shorts-specific ranking, Content ID
Files                Direct-to-object-storage multipart upload
Playback             Manifest + CDN; API never streams bytes
Processing           Async pipeline; encoder is an interface
Feed                 Candidate generation + ranker, not a neural net
Deployment           One region first; CDN is global in concept
Stack                Go, PostgreSQL, Redis, Kafka, object storage,
                     search and CDN behind interfaces

These are planning assumptions, not YouTube’s published numbers. A local mock of S3, CDN, OpenSearch, and FFmpeg is enough to explain the architecture.

2. Functional requirements

The system must:

1. Create a user and a channel.
2. Create a video, upload bytes to object storage, complete the upload.
3. Process the file asynchronously into several resolutions and a thumbnail.
4. Publish and unpublish.
5. Return metadata and a playback manifest URL, not video bytes.
6. Record views without a synchronous database increment.
7. Like and unlike, once per user per video.
8. Comment with cursor pagination.
9. Subscribe to a channel.
10. Store watch history.
11. Search videos by title, description, channel, and tags.
12. Serve a Home feed from mixed candidate sources.

The first version does not include:

• live streaming or DVR;
• ads and auctions;
• real FFmpeg in the demo (the interface must still exist);
• a production CDN;
• a learned recommendation model.

3. Non-functional requirements

Consistency is not one global setting.

Video row, assets, likes, comments    source of truth in PostgreSQL
Object bytes                          object storage
Playback                              CDN cache of those bytes
View count                            Redis, flushed asynchronously
Search document                       derived, eventual
Home feed                             derived, cacheable, stale-OK

Edge cases to keep in mind

• Alice’s client retries upload/complete.
• Two workers pick up UploadCompleted.
• Processing fails after UPLOADED; the original blob remains.
• Bob plays while status is still PROCESSING.
• A viral video turns video:{id}:views into a hot key.
• Duplicate likes from a double tap.
• Comments on a video with millions of rows.
• PRIVATE video in search results.
• Feed cache serves a video Alice unpublished.

4. Estimate the scale

Label every input as an assumption. We need bottlenecks, not a replica of YouTube finance.

Assume a large video site, smaller than today’s YouTube:

Daily active users                 100 million
Watch sessions                     5 per user per day → 500 million/day
Average watch                      10 minutes
Uploads                            500,000 videos/day
Average uploaded file              300 MB
Renditions                         360p, 480p, 720p, 1080p
Transcoded size                    ~2× the mezzanine (rough)
Search                             2 per user per day → 200 million/day
Likes                              50 million/day
Comments                           20 million/day

Playback starts:

500,000,000 / 86,400 ≈ 5,800 average watch starts/s
10× peak             ≈ 58,000/s

Each start also emits a view event. That is already too many UPDATE statements on one videos row if a single clip is hot.

Upload bytes:

500,000 × 300 MB ≈ 150 TB/day ingested
× ~2 for renditions  ≈ 300 TB/day stored (order of magnitude)

Egress if sessions actually stream ~2 Mbps for 10 minutes:

500 million × 600 s × 2 Mbit/s
≈ 6×10^14 bits/day
≈ 75 PB/day

The exact constant is less important than the conclusion: this traffic cannot touch the Go API. CDN and object storage exist because of this number, not as decoration.

What becomes the bottleneck first, in order:

1. CDN / origin egress
2. Transcoding CPU for the upload wave
3. Hot view counters and hot metadata keys
4. Comment and history write partitions for popular videos
5. PostgreSQL for metadata only if we put bytes or view increments there

5. What are we storing?

User / channel     creator identity
Video              metadata, visibility, processing status
Video asset        one rendition or a thumbnail, with object key
Like               (video_id, user_id)
Comment            text, partitioned by video
Subscription       (subscriber_id, channel_id)
Watch history      (user_id, video_id, watched_at, progress)

Bytes live elsewhere:

Original upload     s3://videos/{id}/source
1080p / 720p / …    s3://videos/{id}/1080p/...
master.m3u8         s3://videos/{id}/master.m3u8
thumbnail           s3://videos/{id}/thumb.jpg

Source of truth     PostgreSQL metadata, object storage bytes
Derived             search index, feed cache, CDN
Ephemeral           Redis views, like counts, video:{id} cache
Events              outbox → Kafka

6. APIs

The API speaks JSON. It does not speak MP4.

Create video and upload

POST /v1/videos
Authorization: Bearer <token>
Idempotency-Key: <uuid>

{
  "title": "Weeknight dal",
  "description": "30-minute dal tadka",
  "visibility": "PUBLIC"
}

HTTP 201 Created

{
  "id": "v_8f3a",
  "status": "UPLOADING",
  "upload": {
    "upload_id": "up_91",
    "part_urls": [
      { "part": 1, "url": "https://objects.local/..." }
    ]
  }
}

The client PUTs parts to object storage. Then:

POST /v1/videos/v_8f3a/upload/complete
Authorization: Bearer <token>

HTTP 202 Accepted

{
  "id": "v_8f3a",
  "status": "UPLOADED"
}

202 means processing has been requested. It is not READY.

Playback metadata

GET /v1/videos/v_8f3a

HTTP 200 OK

{
  "id": "v_8f3a",
  "title": "Weeknight dal",
  "status": "PUBLISHED",
  "visibility": "PUBLIC",
  "duration_seconds": 842,
  "thumbnail_url": "https://cdn.example/thumbs/v_8f3a.jpg",
  "playback": {
    "manifest_url": "https://cdn.example/v_8f3a/master.m3u8",
    "resolutions": ["1080p", "720p", "480p", "360p"]
  },
  "view_count": 1284033,
  "like_count": 44120
}

If status is not READY or PUBLISHED, omit playable URLs and return PLAYBACK_NOT_READY when the client asks to play.

Views, likes, comments, feed

POST   /v1/videos/{id}/view
POST   /v1/videos/{id}/like
DELETE /v1/videos/{id}/like
GET    /v1/videos/{id}/comments?cursor=...&limit=20
POST   /v1/videos/{id}/comments
DELETE /v1/comments/{id}
POST   /v1/channels/{id}/subscribe
GET    /v1/search?q=dal
GET    /v1/feed
GET    /v1/users/{id}/history

Errors

400  VALIDATION_ERROR, INVALID_STATE_TRANSITION
401  UNAUTHENTICATED
403  FORBIDDEN, PRIVATE_VIDEO
404  VIDEO_NOT_FOUND
409  ALREADY_LIKED, ALREADY_SUBSCRIBED, UPLOAD_ALREADY_COMPLETED
425  PLAYBACK_NOT_READY   (or 409)

7. Basic data model

users
  id, name, created_at

channels
  id, user_id unique, handle unique, created_at

videos
  id
  creator_id / channel_id
  title, description
  visibility          PUBLIC | PRIVATE | UNLISTED
  status              UPLOADING | UPLOADED | PROCESSING | READY | FAILED | PUBLISHED
  duration_seconds
  thumbnail_key
  processing_version
  version             optimistic lock
  created_at, updated_at, published_at

video_assets
  id, video_id, kind SOURCE|RENDITION|THUMB|MANIFEST
  resolution          nullable
  object_key
  status

likes
  video_id, user_id     unique
  created_at

comments
  id, video_id, user_id, body, created_at
  (video_id, created_at, id)  cursor

subscriptions
  subscriber_id, channel_id    unique
  created_at
  indexes both directions

watch_history
  user_id, video_id, watched_at, progress_seconds

Useful indexes:

videos (channel_id, created_at)
comments (video_id, created_at, id)
likes (video_id, user_id)
subscriptions (subscriber_id, channel_id)
subscriptions (channel_id, subscriber_id)
watch_history (user_id, watched_at)

Likes and subscriptions are uniqueness problems. Unique constraints, not SELECT then INSERT.

Watch history is a write-heavy per-user log. PostgreSQL is fine at the start. The access pattern — user_id partition, watched_at sort — is what later maps onto Cassandra or DynamoDB.

8. Start with metadata, not a streaming server

A first version that POST /videos with a 2 GB multipart body will work on a laptop and fail in production:

API pod buffers the file
load balancer times out
you cannot scale API independently of ingest
you pay CPU twice (ingest + transcode)

So the API only creates a row UPLOADING and returns presigned part URLs. PostgreSQL never sees the bytes.

9. Direct upload

Client                  Video Service              Object storage
  │                          │                            │
  │ POST /videos             │                            │
  │◄──── upload session ─────│                            │
  │                          │                            │
  │ PUT part 1 ──────────────────────────────────────────►│
  │ PUT part 2 ──────────────────────────────────────────►│
  │                          │                            │
  │ POST /upload/complete    │                            │
  │                          │ CompleteMultipart          │
  │                          │ status=UPLOADED            │
  │                          │ outbox UploadCompleted     │

Why multipart: a 2 GB PUT that dies at 90% should resume parts, not restart. The storage interface is:

CreateMultipartUpload
GeneratePresignedPartURL
CompleteMultipartUpload
DeleteObject

Locally this is a filesystem. Later it is S3. The service does not care.

upload/complete is idempotent. If Alice retries after the outbox is written, return the same UPLOADED video. Conditional update:

UPDATE videos SET status = 'UPLOADED'
WHERE id = $id AND status = 'UPLOADING'

Zero rows means a duplicate complete or a bad state; inspect and return 409 or the current body.

Do not publish Kafka inside the transaction. Insert UploadCompleted in the outbox with the status change.

10. Status is a state machine

Illegal jumps must fail.

UPLOADING → UPLOADED → PROCESSING → READY → PUBLISHED
                 │                    │
                 └──► FAILED ◄────────┘

PUBLISHED → READY          unpublish
READY → PUBLISHED          publish (only if READY)

PATCH cannot set PUBLISHED while assets are missing. Publish is its own command after READY.

processing_version increments each processing attempt so a late worker from version 1 cannot overwrite version 2’s renditions.

Visibility is orthogonal:

PUBLIC      search, Home, watch with link
UNLISTED    watch with link; not search/Home
PRIVATE     only the owner

Workers that index search must read visibility at index time and again at query filter time. Unpublish or PRIVATE must drop the document.

11. Processing is a pipeline, not a request

Alice must not wait on FFmpeg.

UploadCompleted
     │
     ▼
Kafka  (key = video_id)
     │
     ▼
Processing worker
     │
     ├─ claim PROCESSING (conditional)
     ├─ Transcoder.Transcode(source) → 1080p, 720p, 480p, 360p
     ├─ thumbnail
     ├─ segment + master.m3u8
     ├─ write video_assets
     └─ READY + outbox VideoProcessingCompleted

Claim:

UPDATE videos
SET status = 'PROCESSING', processing_version = processing_version + 1
WHERE id = $id AND status = 'UPLOADED'

A duplicate Kafka event finds zero rows and stops. That is concurrent-worker safety.

The encoder is an interface. In development it writes fake segment files and a tiny master.m3u8. In production it is FFmpeg or a transcoding fleet. The worker should emit 360p first if you want “playable while still processing”; that is a refinement (status READY with a subset of renditions). First version: all renditions, then READY.

Failures: set FAILED, keep the source object, allow retry which bumps processing_version. Do not delete Alice’s upload because 1080p crashed.

12. Playback is a manifest on a CDN

Adaptive bitrate means the player picks a ladder from a master playlist:

videos/v_8f3a/master.m3u8
  → 1080p/index.m3u8
  → 720p/index.m3u8
  → 480p/index.m3u8
  → 360p/index.m3u8

On a train, the player asks for 360p segments. On Wi-Fi, 1080p. The API does not implement that algorithm. It returns manifest_url.

GET /videos/{id}
  → authorize visibility
  → cache-aside video:{id}
  → CDN.GetPlaybackURL(id)   // signed, short-lived if PRIVATE

The CDN is an edge cache in front of object storage. A local CDN interface can return http://localhost:8080/static/v_8f3a/master.m3u8. Production signs URLs so PRIVATE videos are not a guessable path.

Never io.Copy segment bytes through the video service. That reintroduces the 75 PB/day onto API pods.

13. Views must not lock the video row

POST /videos/v_8f3a/view
  → 202
  → enqueue Viewed event (or write a local buffer)

The handler does not run UPDATE videos SET view_count = view_count + 1. A viral clip would serialize every play on one row.

View API
   ↓
Kafka views  (key = video_id is OK for order; random key is OK for throughput)
   ↓
View worker
   ↓
INCR video:{id}:views
   ↓
every N seconds / N increments
   ↓
flush sum to PostgreSQL

Idempotency: a player retry should not add 50 views. Use (viewer_id or anon_id, video_id, session_id) with a short Redis SETNX TTL, for example 30 seconds. Approximate is correct here. Finance-grade exactness is the wrong bar for a view counter.

Hot key: one Redis key video:{hot}:views concentrates INCR on one shard. Split:

video:{id}:views:0
...
video:{id}:views:15

The client INCR a random shard. Reads SUM the 16 keys (or keep a cached total). Flush writes the sum. This is the same hot-key lesson as a viral short URL or a celebrity cache key.

PostgreSQL view_count is a lagging snapshot for analytics and cold reads. The watch page prefers Redis.

14. Likes are a uniqueness problem

POST /videos/{id}/like

INSERT INTO likes (video_id, user_id) VALUES ($v, $u)
ON CONFLICT DO NOTHING

If inserted, INCR video:{id}:likes and persist periodically, or update an aggregate in the same transaction for low-traffic videos. At YouTube-like like rates, treat like counts like views: unique row in SQL (must not double-like), aggregate in Redis.

DELETE removes the row and DECR, floored at zero. Duplicate POST returns { liked: true, like_count: N } without incrementing.

15. Comments: cursor, partition by video

GET /videos/{id}/comments?cursor=...&limit=20

WHERE video_id = $v
  AND (created_at, id) < ($t, $id)
ORDER BY created_at DESC, id DESC
LIMIT 20

Offset pagination skips and reshuffles as new comments arrive. The cursor is opaque. Storage later shards comments by video_id. A hot video’s comment partition is a known hotspot; we still do not put comments in the videos row.

Delete is authorized: author or channel owner. Soft delete if you need audit; the interview can hard-delete.

16. Subscriptions need both directions

(subscriber_id, channel_id) unique

Alice’s subscriptions page: index (subscriber_id, created_at).
Fan-out of “Alice uploaded”: index (channel_id) to find subscriber ids.

For Home we will not fan-out a new video to 50 million subscriber inboxes (Twitter celebrity problem). We persist the video once and let candidate generation pull recent uploads from subscribed channels.

17. Watch history

POST /users/me/history  { video_id, progress_seconds }
GET  /users/me/history?cursor=...

Key access: latest watches for a user. That is (user_id, watched_at DESC). At 500 million sessions/day, this table outgrows a single Postgres primary before videos does. Design the repository as a log so the backend can move to a wide-column store without changing the API.

Do not join history to video blobs. Hydrate metadata from video:{id} cache.

18. Search is a projection

GET /search?q=dal does not ILIKE 100 million titles in PostgreSQL.

VideoPublished / metadata changed
    → outbox → Kafka → indexer → OpenSearch

Document fields: title, description, channel name, tags, visibility, view_count (lagging), created_at.

Query: text match, filter PUBLIC, rank later with views, freshness, engagement. Locally, an in-memory SearchService is enough. The interface should not assume a neural ranker.

If OpenSearch is down, watch and upload continue. Search returns 503 or empty with a code. PRIVATE documents must not be in the public index.

This is the same search-as-projection idea as hotel search and issue search: OLTP is truth, the index is derived.

19. Home is candidates, then a ranker

GET /feed is not SELECT * FROM videos ORDER BY views DESC.

User
  → Candidate generators (in parallel, bounded)
       subscribed channels' recent uploads
       continue watching
       popular / trending
       similar-to-recent (optional)
  → Filter (seen, PRIVATE, not READY)
  → Ranker (deterministic locally)
  → Page

Local rank can be a linear combination:

score = 0.4 * subscription_boost
      + 0.3 * log(1 + views)
      + 0.2 * freshness
      + 0.1 * completion_rate

That is enough to explain the architecture. A real YouTube ranker is a model. Do not fake one.

Cache feed:{userId} for a minute. Invalidate on subscribe if you are fancy; otherwise short TTL. Stale Home is acceptable. Stale PRIVATE leakage is not: filter on read.

20. Caching

Cache-aside:

video:{id}      metadata + asset list     invalidate on publish/process
channel:{id}
video:{id}:views / shards
video:{id}:likes
feed:{userId}   short TTL

Redis is not the source of truth for likes rows, comments, or video status. If Redis dies, metadata reads go to PostgreSQL, view counts lag or hide, Home is slower. Playback URLs still come from CDN/origin.

21. Idempotency and concurrency

Create-video and complete-upload accept Idempotency-Key because mobile networks retry.

22. Failure scenarios

A failed transcode must not mark the video PUBLISHED and must not delete the source.

23. Observability

Log request_id, video_id, channel_id, status, processing_version, event_id.

Metrics:

upload_complete_total
processing_success / failure
processing_latency
playback_metadata_latency
view_events_total
view_flush_lag
search_latency
feed_latency
cdn is not in-process; watch origin_bytes if you own origin

The product metric for processing is upload-complete to first playable rendition, not worker CPU.

24. How the system grows

10 thousand users. One API, PostgreSQL, local disk, no Kafka. Views can update SQL. This is a weekend project, not the interview answer for “YouTube,” but it is the seed.

1 million users. Presigned upload, one transcoding worker pool, Redis cache, CDN in front of storage, Kafka for processing and views. Single Postgres with replicas.

100 million users. Shard comments and history by video_id / user_id. Shard view counters. Separate processing fleet. Search cluster. Feed ranker as its own service. Isolate hot videos (dedicated Redis slots, origin shields).

1 billion+ users. Multi-region metadata, anycast CDN, transcoding in-region near storage, more aggressive approximate counters, geo-replicated object storage, recommendation as a platform. The interview should stop at mechanisms: partition keys, hot keys, async pipelines, CDN, not a vendor list.

Backpressure: if the processing queue lags, reject or delay new large uploads (quota), do not drop UploadCompleted. Rate-limit view ingest per video and per IP. API pods scale on metadata QPS, which is tiny compared with playback QPS.

25. Where logic lives

cmd/server
internal/video         create, state machine, publish
internal/upload        multipart session, complete
internal/processing    worker, Transcoder interface
internal/playback      manifest URL, authorization
internal/engagement    views, likes, comments
internal/social        subscriptions, history
internal/search
internal/feed          generators + ranker
internal/outbox
internal/storage       ObjectStorage, CDN

A modular monolith. Processing, view aggregation, and indexing are the first pieces to extract when queues and SLOs diverge. Handlers do not call FFmpeg. Workers do not accept 2 GB HTTP bodies.

26. Final architecture

 Client
   │  metadata HTTP
   │  bytes to object storage / from CDN
   ▼
 API  (users, videos, engagement, search, feed)
   │
   ├── PostgreSQL   metadata, likes, comments, subs, outbox
   ├── Redis        cache, view/like counters, feed
   └── Object storage ← multipart upload

 Outbox → Kafka
            ├── Processing workers → Transcoder → assets + READY
            ├── View workers → Redis INCR → flush
            ├── Search indexer
            └── Feed / notification (optional)

 CDN → segments and thumbnails

Upload: API mints URLs → client PUTs → complete → outbox.
Play: API mints manifest URL → player fetches from CDN.
View: event → Redis.
Search/Home: derived, filtered for visibility.

27. Interview-ready summary

Key decisions to remember

1. Scope to upload, process, play, engagement, search, and Home — not live or ads.
2. Files never enter the API process; presigned multipart goes to object storage.
3. Processing is async; encoder is pluggable; claim with a status CAS and processing_version.
4. Playback returns a manifest URL; ABR is the player’s job; bytes are CDN.
5. Views are events + Redis (sharded INCR) + periodic flush, not row locks.
6. Likes and subscriptions are unique constraints; aggregates can be cached.
7. Comments and history use cursors and partition keys, not OFFSET.
8. Search and Home are projections; filter visibility on index and on read.
9. Home is candidate generation plus a ranker, not ORDER BY views.
10. Redis is not the video store; object storage is not the like store.

Interview cheat sheet

The ten points to say out loud:

1. Control plane vs data plane. Metadata in the API; bytes in storage/CDN.
2. Upload complete is a state transition plus an outbox event.
3. Transcoding is a pipeline with idempotent workers.
4. ABR is multiple renditions + master.m3u8, not one MP4 from the API.
5. View counts are approximate, sharded, and asynchronous on purpose.
6. Hot keys (viral video) need sharded counters and cached metadata.
7. Unique likes/subs in the database, not in application memory.
8. Search lag must not block watch; PRIVATE must not leak.
9. Feed = generate many, rank few, cache briefly.
10. Scale playback with CDN; scale uploads with storage; scale views with streams; scale API last.

Likely interviewer follow-up questions

• Why not upload through the API with a streaming body?
• How do you resume a failed 2 GB upload?
• When is a video first playable if 1080p is still encoding?
• How do you stop two workers from transcoding the same source?
• Why is a slightly wrong view count acceptable?
• How do sharded counters merge on read?
• How do you hide an unpublished video that is still in a cached feed?
• How is this Home feed different from Twitter’s fan-out?
• What happens if the CDN origin is the same disk as uploads?
• How would live streaming change the pipeline?

Senior-level points that differentiate the answer

• Separate Alice’s durable source object from derived renditions.
• Use conditional updates for UPLOADING → UPLOADED → PROCESSING → READY.
• Treat view accuracy as an SLO, not as a bank ledger.
• Pre-empt the viral-video hot key before it appears in Q&A.
• Filter visibility twice: index and serving.
• Keep the ranker replaceable; do not ship a fake neural net.
• Quote egress math to justify the CDN in one sentence.
• Evolve comments/history storage from Postgres to a log-shaped store without changing the API.

A 1–2 minute verbal answer

I would scope YouTube to upload, async processing, CDN playback, approximate views, likes, comments, subscriptions, search, and a candidate-generation Home feed. At 100 million DAU and 500 million watches a day, playback starts are tens of thousands per second at peak and egress is petabytes, so the API must never stream video. Uploads use presigned multipart URLs into object storage. Complete is an idempotent status change that writes an outbox event. Workers claim PROCESSING, transcode 360p–1080p through an encoder interface, write a master HLS manifest, and mark READY. The watch endpoint returns metadata and a CDN manifest URL.

Views go to Kafka and sharded Redis counters, flushed later to Postgres. Likes are a unique (video_id, user_id). Comments paginate by cursor on (video_id, created_at). Search and Home are derived: index on publish, generate candidates from subscriptions and popularity, then rank. If Redis, Kafka, or search is down, processing and watch degrade independently; a failed transcode does not delete the source.

For the broader interview framework around this problem, see the System Design Interview Complete Guide.