System design · Concept 2
Back-of-the-Envelope Estimation
Latency numbers, powers of two and quick QPS, storage and bandwidth math so your design decisions rest on numbers instead of vibes.
In short
- Round hard: a day is about 100K seconds
- Average, then peak, for reads and writes
- Storage includes retention, replication and indexes
- Bandwidth in bits: multiply bytes by 8
- End every number with a decision
Estimates aren't about precision. They tell you which order of magnitude you're in, because that decides the architecture: one database or fifty shards, a cache or no cache, a CDN or app servers serving files. Round aggressively, state your assumptions, and end every calculation with a decision.
Powers of two and units
| Power | Exact value | Approx | Unit |
|---|---|---|---|
| 2^10 | 1,024 | 1 thousand | 1 KB |
| 2^20 | 1,048,576 | 1 million | 1 MB |
| 2^30 | 1,073,741,824 | 1 billion | 1 GB |
| 2^40 | ~1.1 × 10^12 | 1 trillion | 1 TB |
| 2^50 | ~1.1 × 10^15 | 1 quadrillion | 1 PB |
Handy conversions:
- 1 day = 86,400 s ≈ 10^5 s. Using 100K makes mental math easy and slightly underestimates QPS, so add headroom.
- 1 month ≈ 2.6M s, 1 year ≈ 31.5M s ≈ 3 × 10^7 s
- 1M requests/day ≈ 12 req/s, 100M/day ≈ 1,200 req/s, 1B/day ≈ 12K req/s
- 8 bits = 1 byte. Network links are quoted in bits (1 Gbps ≈ 125 MB/s).
- Base62 with 7 characters gives 62^7 ≈ 3.5 × 10^12 combinations. 2^32 ≈ 4.3 billion is the limit of a 32-bit ID.
Latency numbers worth memorizing
Approximate figures. The ratios between them matter more than the exact values.
| Operation | Approx time |
|---|---|
| L1 cache reference | 1 ns |
| L2 cache reference | 4 ns |
| Main memory reference | 100 ns |
| Compress 1 KB (fast codec) | 2 µs |
| Read 1 MB sequentially from memory | 5-10 µs |
| Random 4 KB read from NVMe SSD | 20-100 µs |
| Read 1 MB sequentially from SSD | 0.2-1 ms |
| Round trip in the same datacenter / AZ | 0.2-0.5 ms |
| Redis GET over the network | 0.3-1 ms |
| Simple indexed SQL query | 1-5 ms |
| Round trip across AZs in one region | 1-2 ms |
| HDD seek | 5-10 ms |
| Read 1 MB sequentially from HDD | 5-10 ms |
| Round trip US East ↔ US West | 60-70 ms |
| Round trip US ↔ Europe | 80-150 ms |
| New TLS connection | 1-2 extra round trips |
What this means for design:
- Memory is ~100-1000x faster than SSD, which is why caches pay off.
- In-datacenter hops are cheap (sub-millisecond), but five sequential service calls at p99 add up fast.
- Cross-region round trips are 50-150 ms. Never put them in a loop on the request path. Replicate data closer to users instead.
- Reuse connections (keep-alive, pooling) to avoid paying TLS handshakes on every request.
Throughput rules of thumb (per node, order of magnitude)
| Component | Rough capacity |
|---|---|
| Stateless API server (simple JSON CRUD) | 1K-10K req/s |
| PostgreSQL/MySQL primary | 10K-50K simple reads/s, 1K-10K writes/s |
| Redis (single shard) | 100K+ ops/s |
| Cassandra / DynamoDB-style node | ~10K writes/s |
| Kafka partition | ~10 MB/s; a broker handles hundreds of MB/s |
| WebSocket gateway | 50K-500K idle connections per node (memory-bound) |
| 1 Gbps NIC | ~125 MB/s; 10 Gbps ≈ 1.25 GB/s |
These numbers are for planning. Real capacity depends on payload size, query shape and hardware, so benchmark before committing.
The core formulas
avg QPS = requests per day / 86,400
peak QPS = avg QPS × 2-3 (spiky events: × 10 or more)
storage = items per day × item size × retention days × replication factor
bandwidth = QPS × payload size
cache memory = hot fraction (often ~20%) × daily working set
instances needed = peak QPS / (per-instance QPS × target utilization ~0.6), then N+1 per AZ
Worked example 1: photo-sharing app
Assumptions: 50M DAU. Each user uploads 0.2 photos/day and views 30. Originals average 2 MB, and a feed-size image is 200 KB.
- Writes: 50M × 0.2 = 10M uploads/day → 10M / 86,400 ≈ 115/s avg, ~350/s peak
- Reads: 50M × 30 = 1.5B views/day → ~17K/s avg, ~50K/s peak
- Read:write ≈ 150:1 → read-heavy, so a CDN and caching are essential
- Storage: 10M × 2 MB = 20 TB/day → ~7 PB/year before replication → object storage with lifecycle tiering. A database can't do this.
- Egress: 17K/s × 200 KB ≈ 3.4 GB/s ≈ 27 Gbps average → serve images from a CDN, never from app servers
- Metadata: 10M rows/day × 500 B = 5 GB/day ≈ 1.8 TB/year → one SQL primary handles it for a while. Plan partitioning by year 2-3.
Worked example 2: does it fit on one database?
A SaaS product has 1B rows of 500 bytes each.
- Raw data: 1B × 500 B = 500 GB. Indexes often double that, so ~1 TB.
- Modern managed SQL instances handle several TB with NVMe storage, so this fits one primary.
- Writes: 2K/s → fine for one primary. Reads: 30K/s → add 2-3 read replicas plus a cache.
- Conclusion: no sharding yet. Revisit once you approach ~5-10 TB, sustained writes above ~10K/s, or when vacuum and backup times hurt.
Worked example 3: cache sizing
An API serves 20K reads/s over 10M distinct items of 1 KB each per day.
- Full working set: 10M × 1 KB = 10 GB. Caching the hottest 20% takes 2 GB, but the whole set fits in one Redis node.
- With an 80-90% hit ratio, the DB sees 2K-4K reads/s instead of 20K.
- Add a replica for availability. Shard the cache only for throughput (>100K ops/s) or memory (>~50-100 GB).
Worked example 4: how many servers?
Peak load is 50K req/s, and one instance handles 2K req/s at 100% CPU. You want to run at 60%.
- 50K / (2K × 0.6) ≈ 42 instances
- Across 3 AZs that's 14 per AZ. Add one AZ's worth of headroom for an AZ failure (N+1 at the AZ level): 21 per AZ → ~63 instances at peak, with autoscaling down at night.
Worked example 5: bandwidth and messages
A chat app has 50M DAU, each sending 40 messages of 200 bytes per day.
- 2B messages/day → ~23K/s avg, ~70K/s peak
- Ingest bandwidth: 70K × 200 B = 14 MB/s. Small; bandwidth isn't the bottleneck here.
- Storage: 2B × 200 B = 400 GB/day → ~150 TB/year raw, ~450 TB with 3x replication → a horizontally scalable store (wide-column).
- Connections: if 40% are online at peak, that's 20M concurrent sockets → at 100K per node, ~200 gateway servers.
Presenting estimates
- Write them down on the board, in the notes or the design doc.
- Round freely: 86,400 → 100K, 2.6M → 3M. Stay within one order of magnitude.
- Separate metadata from blobs. They live in different stores and scale differently.
- State the conclusion: "≈ 1 TB, fits one node", "≈ 30 Gbps, needs a CDN".
Common mistakes:
- Forgetting peak vs average (2-10x)
- Forgetting the replication factor (usually 3x) and index overhead
- Mixing up bits and bytes (Gbps vs GB/s)
- Ignoring fan-out amplification: one post sent to 200 followers is 200 writes
- Computing numbers you never use
When to use / trade-offs
- Use estimates early, right after requirements, and again during deep dives ("how big is this cache?").
- Spend 3-5 minutes, not 15. If a number doesn't change a decision, skip it.
- Trade-off: precision vs speed. Being 2x off is fine. Being 1000x off (TB vs PB) leads to the wrong architecture.
- At work, replace guesses with real metrics as soon as they exist, and re-estimate before big launches.
Checklist
- DAU → requests/day → avg and peak QPS for reads and writes
- Read:write ratio and what it implies
- Storage per day and per year, including replication
- Bandwidth (ingress/egress) and whether a CDN is needed
- Memory for caches
- Node counts for the busiest tier
Practise this concept
In the app: a short video and a 11-question concept check. Free with a GitHub account.
Open in the app