System design · Concept 2

Back-of-the-Envelope Estimation

Latency numbers, powers of two and quick QPS, storage and bandwidth math so your design decisions rest on numbers instead of vibes.

7 min read

In short

  • Round hard: a day is about 100K seconds
  • Average, then peak, for reads and writes
  • Storage includes retention, replication and indexes
  • Bandwidth in bits: multiply bytes by 8
  • End every number with a decision

Estimates aren't about precision. They tell you which order of magnitude you're in, because that decides the architecture: one database or fifty shards, a cache or no cache, a CDN or app servers serving files. Round aggressively, state your assumptions, and end every calculation with a decision.

Powers of two and units

PowerExact valueApproxUnit
2^101,0241 thousand1 KB
2^201,048,5761 million1 MB
2^301,073,741,8241 billion1 GB
2^40~1.1 × 10^121 trillion1 TB
2^50~1.1 × 10^151 quadrillion1 PB

Handy conversions:

  • 1 day = 86,400 s ≈ 10^5 s. Using 100K makes mental math easy and slightly underestimates QPS, so add headroom.
  • 1 month ≈ 2.6M s, 1 year ≈ 31.5M s ≈ 3 × 10^7 s
  • 1M requests/day ≈ 12 req/s, 100M/day ≈ 1,200 req/s, 1B/day ≈ 12K req/s
  • 8 bits = 1 byte. Network links are quoted in bits (1 Gbps ≈ 125 MB/s).
  • Base62 with 7 characters gives 62^7 ≈ 3.5 × 10^12 combinations. 2^32 ≈ 4.3 billion is the limit of a 32-bit ID.

Latency numbers worth memorizing

Approximate figures. The ratios between them matter more than the exact values.

OperationApprox time
L1 cache reference1 ns
L2 cache reference4 ns
Main memory reference100 ns
Compress 1 KB (fast codec)2 µs
Read 1 MB sequentially from memory5-10 µs
Random 4 KB read from NVMe SSD20-100 µs
Read 1 MB sequentially from SSD0.2-1 ms
Round trip in the same datacenter / AZ0.2-0.5 ms
Redis GET over the network0.3-1 ms
Simple indexed SQL query1-5 ms
Round trip across AZs in one region1-2 ms
HDD seek5-10 ms
Read 1 MB sequentially from HDD5-10 ms
Round trip US East ↔ US West60-70 ms
Round trip US ↔ Europe80-150 ms
New TLS connection1-2 extra round trips

What this means for design:

  • Memory is ~100-1000x faster than SSD, which is why caches pay off.
  • In-datacenter hops are cheap (sub-millisecond), but five sequential service calls at p99 add up fast.
  • Cross-region round trips are 50-150 ms. Never put them in a loop on the request path. Replicate data closer to users instead.
  • Reuse connections (keep-alive, pooling) to avoid paying TLS handshakes on every request.

Throughput rules of thumb (per node, order of magnitude)

ComponentRough capacity
Stateless API server (simple JSON CRUD)1K-10K req/s
PostgreSQL/MySQL primary10K-50K simple reads/s, 1K-10K writes/s
Redis (single shard)100K+ ops/s
Cassandra / DynamoDB-style node~10K writes/s
Kafka partition~10 MB/s; a broker handles hundreds of MB/s
WebSocket gateway50K-500K idle connections per node (memory-bound)
1 Gbps NIC~125 MB/s; 10 Gbps ≈ 1.25 GB/s

These numbers are for planning. Real capacity depends on payload size, query shape and hardware, so benchmark before committing.

The core formulas

avg QPS          = requests per day / 86,400
peak QPS         = avg QPS × 2-3   (spiky events: × 10 or more)
storage          = items per day × item size × retention days × replication factor
bandwidth        = QPS × payload size
cache memory     = hot fraction (often ~20%) × daily working set
instances needed = peak QPS / (per-instance QPS × target utilization ~0.6), then N+1 per AZ

Worked example 1: photo-sharing app

Assumptions: 50M DAU. Each user uploads 0.2 photos/day and views 30. Originals average 2 MB, and a feed-size image is 200 KB.

  • Writes: 50M × 0.2 = 10M uploads/day → 10M / 86,400 ≈ 115/s avg, ~350/s peak
  • Reads: 50M × 30 = 1.5B views/day → ~17K/s avg, ~50K/s peak
  • Read:write ≈ 150:1 → read-heavy, so a CDN and caching are essential
  • Storage: 10M × 2 MB = 20 TB/day → ~7 PB/year before replication → object storage with lifecycle tiering. A database can't do this.
  • Egress: 17K/s × 200 KB ≈ 3.4 GB/s ≈ 27 Gbps average → serve images from a CDN, never from app servers
  • Metadata: 10M rows/day × 500 B = 5 GB/day ≈ 1.8 TB/year → one SQL primary handles it for a while. Plan partitioning by year 2-3.

Worked example 2: does it fit on one database?

A SaaS product has 1B rows of 500 bytes each.

  • Raw data: 1B × 500 B = 500 GB. Indexes often double that, so ~1 TB.
  • Modern managed SQL instances handle several TB with NVMe storage, so this fits one primary.
  • Writes: 2K/s → fine for one primary. Reads: 30K/s → add 2-3 read replicas plus a cache.
  • Conclusion: no sharding yet. Revisit once you approach ~5-10 TB, sustained writes above ~10K/s, or when vacuum and backup times hurt.

Worked example 3: cache sizing

An API serves 20K reads/s over 10M distinct items of 1 KB each per day.

  • Full working set: 10M × 1 KB = 10 GB. Caching the hottest 20% takes 2 GB, but the whole set fits in one Redis node.
  • With an 80-90% hit ratio, the DB sees 2K-4K reads/s instead of 20K.
  • Add a replica for availability. Shard the cache only for throughput (>100K ops/s) or memory (>~50-100 GB).

Worked example 4: how many servers?

Peak load is 50K req/s, and one instance handles 2K req/s at 100% CPU. You want to run at 60%.

  • 50K / (2K × 0.6) ≈ 42 instances
  • Across 3 AZs that's 14 per AZ. Add one AZ's worth of headroom for an AZ failure (N+1 at the AZ level): 21 per AZ → ~63 instances at peak, with autoscaling down at night.

Worked example 5: bandwidth and messages

A chat app has 50M DAU, each sending 40 messages of 200 bytes per day.

  • 2B messages/day → ~23K/s avg, ~70K/s peak
  • Ingest bandwidth: 70K × 200 B = 14 MB/s. Small; bandwidth isn't the bottleneck here.
  • Storage: 2B × 200 B = 400 GB/day → ~150 TB/year raw, ~450 TB with 3x replication → a horizontally scalable store (wide-column).
  • Connections: if 40% are online at peak, that's 20M concurrent sockets → at 100K per node, ~200 gateway servers.

Presenting estimates

  • Write them down on the board, in the notes or the design doc.
  • Round freely: 86,400 → 100K, 2.6M → 3M. Stay within one order of magnitude.
  • Separate metadata from blobs. They live in different stores and scale differently.
  • State the conclusion: "≈ 1 TB, fits one node", "≈ 30 Gbps, needs a CDN".

Common mistakes:

  • Forgetting peak vs average (2-10x)
  • Forgetting the replication factor (usually 3x) and index overhead
  • Mixing up bits and bytes (Gbps vs GB/s)
  • Ignoring fan-out amplification: one post sent to 200 followers is 200 writes
  • Computing numbers you never use

When to use / trade-offs

  • Use estimates early, right after requirements, and again during deep dives ("how big is this cache?").
  • Spend 3-5 minutes, not 15. If a number doesn't change a decision, skip it.
  • Trade-off: precision vs speed. Being 2x off is fine. Being 1000x off (TB vs PB) leads to the wrong architecture.
  • At work, replace guesses with real metrics as soon as they exist, and re-estimate before big launches.

Checklist

  • DAU → requests/day → avg and peak QPS for reads and writes
  • Read:write ratio and what it implies
  • Storage per day and per year, including replication
  • Bandwidth (ingress/egress) and whether a CDN is needed
  • Memory for caches
  • Node counts for the busiest tier

Practise this concept

In the app: a short video and a 11-question concept check. Free with a GitHub account.

Open in the app