A Full Architecture Design Round
One question worked end to end — clarify, size, sketch, fail it, cost it, trade off — with the weak version of each step shown alongside the strong one.
The question: “Design the backend for a photo-sharing app. Users upload photos, follow other users, and see a feed of photos from people they follow.”
Forty-five minutes. Here is the whole round, with the weak version of each step shown next to the strong one.
Step 1 — clarify (5 min)
WEAK: "Okay, so we'll need a load balancer, some app servers, S3 for the
photos, and a database. Let me draw that."
STRONG: "Before I draw — how many users, and how photo-heavy is this?
Is the feed chronological or ranked?
And is there a hard latency target for feed load?"The answers, as given:
100M registered, 20M daily active
each user posts ~1 photo/day, views ~100 feed items/day
photos average 3 MB, thumbnails needed
feed is chronological (ranked is a later phase)
feed p99 under 200 ms
some users have 10M+ followersThe last line is the whole problem. A celebrity with 10 million followers is what makes this question interesting rather than routine, and noticing it in the clarification phase is worth saying out loud:
“The 10M-follower case is going to drive the design — a naive fan-out on write means one post becomes ten million writes. I’ll come back to that.”
Step 2 — size it (5 min)
# sizing.py
DAU, POSTS_PER_DAY, VIEWS_PER_DAY = 20_000_000, 1, 100
PHOTO_MB, THUMB_MB = 3, 0.15
SEC_PER_DAY, PEAK = 86_400, 3
writes_day = DAU * POSTS_PER_DAY
reads_day = DAU * VIEWS_PER_DAY / 20 # ~20 items per feed request
print(f"write rps avg {writes_day/SEC_PER_DAY:>10,.0f} peak {writes_day/SEC_PER_DAY*PEAK:>10,.0f}")
print(f"read rps avg {reads_day/SEC_PER_DAY:>10,.0f} peak {reads_day/SEC_PER_DAY*PEAK:>10,.0f}")
print(f"read:write ratio {reads_day/writes_day:>8,.0f}:1\n")
daily_gb = writes_day * (PHOTO_MB + THUMB_MB) / 1024
print(f"new storage {daily_gb:>10,.0f} GB/day {daily_gb*365/1024:>8,.0f} TB/year")
print(f"upload bandwidth peak {writes_day/SEC_PER_DAY*PEAK*PHOTO_MB:>8,.0f} MB/s")
print(f"serving bandwidth peak {reads_day/SEC_PER_DAY*PEAK*20*THUMB_MB:>7,.0f} MB/s (thumbnails)")$ python sizing.py
write rps avg 231 peak 694
read rps avg 11,574 peak 34,722
read:write ratio 50:1
new storage 61,523 GB/day 21,930 TB/year
upload bandwidth peak 2,083 MB/s
serving bandwidth peak 20,833 MB/s (thumbnails)Four things now decided by arithmetic rather than preference:
- 50:1 read:write. This is a read-optimised system. Caching and denormalisation are justified; write-path cleverness mostly is not.
- 22 PB a year. Object storage with lifecycle tiering, not block storage, and not a database.
- 20 GB/s peak serving bandwidth, ~7 GB/s average. A CDN is not optional — from the storage lesson, 17 PB a month served straight from origin is roughly $1.5M in egress.
- 694 writes/sec peak is small. A single well-provisioned database handles it. The write path is not the hard part; fan-out is.
Step 3 — sketch it (10 min)
┌─────────┐
mobile / web ───────▶│ CDN │────▶ thumbnails, photos (signed URLs)
└────┬────┘
│ miss
┌────▼────┐
│ API │ ALB → containers, autoscaled
│ gateway │
└────┬────┘
┌──────────────────┼──────────────────┐
▼ ▼ ▼
┌────────────┐ ┌──────────────┐ ┌──────────────┐
│ UPLOAD │ │ FEED │ │ SOCIAL │
│ service │ │ service │ │ service │
└─────┬──────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
presigned PUT ┌────▼────┐ ┌──────▼──────┐
│ │ Redis │ │ Postgres │
▼ │ feed │ │ users, │
┌──────────┐ │ cache │ │ follows │
│ Object │ └────▲────┘ └──────┬──────┘
│ storage │ │ │
└────┬─────┘ ┌────┴──────────────────▼──────┐
│ event │ FAN-OUT WORKERS │
└─────────────▶│ queue → write to followers │
└──────────────────────────────┘Five decisions in that picture, each worth one sentence:
- Presigned upload URLs. The client PUTs directly to object storage; the 3 MB never touches the application tier. That removes 2 GB/s of ingress from the fleet and is the single highest- leverage decision on the upload path.
- Object-storage event triggers thumbnailing, asynchronously. The user’s upload completes before the thumbnail exists, and the client shows a placeholder.
- Feed is precomputed into a cache, because 50:1 reads.
- Follows in a relational store. The graph is small per user, needs consistency, and the query patterns are simple.
- Fan-out is a queue, not a synchronous write.
Step 4 — the hard part: fan-out
# fanout.py
FOLLOWERS = {"typical": 200, "popular": 50_000, "celebrity": 10_000_000}
WRITE_MS, READ_MS_PER_SOURCE = 0.5, 0.4
print(f"{'user type':<12} {'followers':>12} {'fan-out on WRITE':>20} {'fan-out on READ':>18}")
for kind, n in FOLLOWERS.items():
write_cost = n * WRITE_MS / 1000
read_cost = 200 * READ_MS_PER_SOURCE # reader follows ~200 people
print(f"{kind:<12} {n:>12,} {write_cost:>17,.1f} s {read_cost:>15,.0f} ms")$ python fanout.py
user type followers fan-out on WRITE fan-out on READ
typical 200 0.1 s 80 ms
popular 50,000 25.0 s 80 ms
celebrity 10,000,000 5,000.0 s 80 msFan-out on write for a celebrity is 83 minutes of writes for one post. Fan-out on read is constant regardless of the poster, and costs the reader 80 ms every time.
Neither alone works, and saying that is the answer:
FAN-OUT ON WRITE push the post into every follower's feed at post time
read: O(1) — the feed is already built. Fast.
write: O(followers) — unusable for celebrities.
waste: builds feeds for inactive users who never look.
FAN-OUT ON READ gather from everyone the reader follows at read time
read: O(following) — 200 queries, or one query over 200 sources.
write: O(1). Trivial.
problem: pays the cost on the hot path, 50:1 in the wrong direction.
HYBRID push for normal users, pull for celebrities
a reader's feed = their precomputed timeline
MERGED WITH a live pull from the few celebrities
they follow, which is cacheable globally“I’d fan out on write for anyone under, say, ten thousand followers — that is 5 seconds of writes, fine asynchronously — and mark accounts above that threshold as pull. A reader’s feed then merges their precomputed timeline with a live read of the celebrities they follow. The celebrity’s own timeline is one cache entry serving millions of readers, so the pull is cheap. The threshold is a tunable, and I’d want to measure the follower distribution before picking it.”
Two refinements to volunteer if there is time:
- Only fan out to active users. Someone who has not opened the app in ninety days does not need a precomputed feed; build it lazily when they return. On these numbers that is most of the 100M registered against 20M daily active — an 80% reduction in fan-out writes.
- Cap the precomputed feed. Store the most recent 500 entries per user, not all history. Older pages fall back to a query. That bounds the cache at 20M × 500 × ~50 bytes ≈ 500 GB.
Step 5 — fail it (5 min)
COMPONENT FAILS EFFECT MITIGATION
CDN edge outage latency up, origin load multi-CDN or
spikes origin autoscale
object storage regional outage no photos at all cross-region
(uploads and views) replication
feed cache (Redis) node loss feeds rebuild from DB; replicas; and
read amplification rebuild is the
risk, not the loss
fan-out workers backlog feeds go stale; it degrades, not
posts still succeed breaks — good
Postgres primary failover follows/unfollows fail multi-AZ, 30-120s
~1 min; feeds keep
serving from cache
whole region outage total depends on the
stated RTOThe row worth pointing at is the fan-out workers: a backlog makes feeds stale but does not stop posting or reading. That is a designed-in degradation, and naming it as deliberate is different from listing it as a risk.
The dangerous row is the feed cache. From the availability lesson, losing a cache sized to absorb a 50:1 read ratio means the database suddenly receives 50x its normal read load:
“If the feed cache is lost, the database gets fifty times its normal reads. I’d want request coalescing so a thousand simultaneous misses for the same feed become one query, and I’d size the database to survive a partial cache loss rather than assuming the cache is always there.”
Step 6 — cost it (5 min)
# cost.py
TB = 1024
SEC_PER_MONTH = 86_400 * 30
# average, not peak — peak was 20,833 MB/s, average is a third of that
serving_gb_month = 6_944 * SEC_PER_MONTH / 1024
stored_gb = 21_930 * TB / 2 # ~6 months accumulated
CDN_GB, ORIGIN_GB, CDN_HIT = 0.020, 0.085, 0.95
storage = stored_gb * 0.023
cdn = serving_gb_month * CDN_GB
origin = serving_gb_month * (1 - CDN_HIT) * ORIGIN_GB
compute = 60 * 60.74 + 40 * 121.48 # api tier + fan-out workers
database = 2 * 1_200 # multi-AZ primary + replica
cache = 6 * 780 # redis cluster
lines = [("storage", storage), ("CDN", cdn), ("origin egress", origin),
("compute", compute), ("database", database), ("cache", cache)]
total = sum(v for _, v in lines)
for k, v in lines:
print(f"{k:<15} ${v:>12,.0f} {100*v/total:>5.1f}%")
print(f"{'TOTAL':<15} ${total:>12,.0f}")$ python cost.py
storage $ 258,248 36.9%
CDN $ 351,540 50.2%
origin egress $ 74,702 10.7%
compute $ 8,504 1.2%
database $ 2,400 0.3%
cache $ 4,680 0.7%
TOTAL $ 700,074Data movement is 61% and storage is 37%. Compute, database and cache together are 2.2%.
That inversion is the finding, and it redirects the whole optimisation conversation:
serve WebP/AVIF instead of JPEG 30-50% smaller → cuts BOTH the CDN line
and future storage growth. The largest lever.
size thumbnails to the device do not send 1080p into a 320px slot
raise the CDN hit rate to 99% origin egress $74.7k → $14.9k
negotiate committed CDN pricing at 17 PB/month, list price is not the price
tier photos older than 90 days storage is 37% — genuinely worth doing here
rightsize the instances 1.2% of the bill. Not worth the meeting.Note the last two rows against each other. Storage tiering is often the intuitive saving and is often negligible — here it is the second-biggest line and is worth real effort. Instance rightsizing is the reflex answer in most cost conversations and would save at most a few thousand dollars against a $700k bill.
Saying which optimisation not to bother with, and why, is as valuable as naming the ones that matter. It demonstrates that the percentages drove the answer rather than habit.
Step 7 — trade off (5 min)
CHOSE GAVE UP WOULD REVISIT IF
hybrid fan-out simplicity; two code paths follower distribution
were flatter
precomputed feed in a cache consistency — a new post takes seconds
to appear strict real-time required
chronological feed engagement (ranked does product asks for ranking;
better) that changes the read path
presigned direct upload no server-side validation abuse or malware scanning
before the object lands becomes a requirement
async thumbnailing a brief window with no thumb users complain about
the placeholder
relational store for follows harder to scale writes follow-graph writes grow
past one primary past ~10k/secA design with no “gave up” column is not finished. Volunteering this table unprompted is the clearest possible signal that you have designed systems rather than read about them.
What the interviewer wrote down
✓ noticed the celebrity case during clarification, before designing
✓ arithmetic out loud; 50:1 ratio drove the design rather than being decorative
✓ presigned upload — kept 2 GB/s off the application tier
✓ named fan-out on write vs read, then hybridised with a stated threshold
✓ active-user optimisation and feed cap volunteered, not prompted
✓ cache loss identified as the dangerous failure, with request coalescing
✓ costed it; identified compute rightsizing as not worth doing, and image
format as the lever that cuts two lines at once
✓ trade-off table with what would change each decision
△ did not cover: auth, abuse/moderation, GDPR deletion, search, notifications
— said so explicitly and named which to do nextThat last line is scope management, not failure. Saying “I have not covered moderation or deletion-on-request; deletion is the one I would do next because it changes the storage design” reads completely differently from silence.
The template, portable to any question
1. CLARIFY scale, availability target, hardest constraint.
Two or three questions. Say which answer changes the design.
2. SIZE requests/sec, GB/day, read:write ratio. Out loud.
The ratio decides more than any service choice.
3. SKETCH boxes and labelled arrows. Where state lives.
One sentence justifying each decision.
4. HARD PART every question has one. Find it and go deep.
Name the two naive approaches and why you hybridised.
5. FAIL IT per component: what dies, who notices, what recovers.
Name the deliberate degradations as deliberate.
6. COST IT rough monthly figure, and which line dominates.
Name an optimisation that is NOT worth doing, and why.
7. TRADE OFF what you gave up, and what would make you revisit.
Then say what you did not cover.Practice
1. Start drawing before asking about follower distribution.
You design fan-out on write, and the celebrity case invalidates it at minute 30.The 10M-follower line is the whole problem. Noticing it during clarification, not after, is what the step is for.
2. Compute the read:write ratio before choosing services.
50:1 — a read-optimised system. Caching and denormalisation are justified.The ratio decides more than any individual service choice, and it takes thirty seconds.
3. Route 3 MB uploads through the application tier.
2 GB/s of ingress the fleet has to absorb, for no benefit.Presigned URLs let the client PUT straight to object storage. The highest-leverage decision on the upload path.
4. Propose rightsizing the instances to save money.
Compute is 1.2% of a $700k bill. Data movement is 61% and storage is 37%.The reflex answer saves a few thousand dollars. Read the percentages first, and say which optimisation is not worth the meeting.
Next: infrastructure as code — the hands-on round, and what a Terraform plan reveals.