System design · Concept 10
Cloud Building Blocks
Design with generic boxes, then name the service that plays each role and say why. Maps every board component to AWS, Azure, Google Cloud and self-hosted options, and covers the decisions that recur in hosting, edge, identity, data, messaging, observability, resilience and cost.
In short
- Generic boxes first; name a service only when it matters
- Every box maps to AWS, Azure, Google Cloud and self-hosted
- Provider choice: skills, contracts, managed vs self-hosted, cost, lock-in
- Queue for commands, pub/sub for facts, stream for volume
- Platform identities, a cached secrets store, RPO and RTO
Design with generic boxes first (a load balancer, a queue, a cache, a SQL database). Only when it matters, name the service that plays each role and say why that one. Picking a provider is a trade-off, never a default:
- Team skills: the cloud your team already operates well beats a slightly better product nobody knows.
- Existing contracts: committed spend, enterprise agreements and compliance approvals are real constraints.
- Managed vs self-hosted: a managed service trades control and some portability for backups, patching and failover you don't run yourself.
- Cost: list prices matter less than how a service bills (per request, per hour of provisioned capacity, per GB moved).
- Lock-in: open protocols and engines (PostgreSQL, Redis, Kafka, OpenTelemetry, Kubernetes) keep the exit door open; proprietary APIs make it narrower. Lock-in is a cost to weigh, not a sin.
Board component → services
Provider order is alphabetical (AWS, Azure, Google Cloud), then Other (self-hosted software or a third party).
| Board component | AWS | Azure | Google Cloud | Other | Notes for a design review |
|---|---|---|---|---|---|
| client | Amplify Hosting, or S3 + CloudFront | Static Web Apps, or Blob Storage + Front Door | Firebase Hosting, or Cloud Storage + Cloud CDN | Vercel, Netlify | The client is your code; the cloud only hosts the files. SPAs need a fallback to index.html so deep links reach the router |
| dns | Route 53 | Azure DNS, Traffic Manager | Cloud DNS | Cloudflare DNS, NS1 | All three offer latency, geo or failover routing; DNS failover is bounded by TTLs |
| cdn | CloudFront | Front Door | Cloud CDN | Cloudflare, Fastly, Akamai | Front Door is Azure's CDN; Cloud CDN is switched on behind Google's external Application Load Balancer |
| load-balancer | Application Load Balancer (L7), Network Load Balancer (L4) | Front Door (global L7), Application Gateway (regional L7), Load Balancer (L4) | Cloud Load Balancing (Application LB L7, Network LB L4) | NGINX, HAProxy, Envoy | Google's external Application LB can be global behind one anycast IP; AWS load balancers are regional, with CloudFront or Global Accelerator in front for a global entry |
| api-gateway | API Gateway | API Management | API Gateway, Apigee | Kong, Tyk, Envoy | JWT validation, rate limits and quotas, versions. Apigee and API Management add a developer portal |
| realtime | API Gateway WebSocket APIs, AppSync | SignalR Service, Web PubSub | Cloud Run (WebSockets), Firebase Realtime Database | Socket.IO, Ably, Pusher | Google Cloud has no dedicated WebSocket gateway: run your own on Cloud Run or use Firebase. Connections have a maximum lifetime everywhere, so clients must reconnect |
| service | ECS on Fargate, EKS, Lambda, Elastic Beanstalk | Container Apps, App Service, AKS, Functions | Cloud Run, GKE, Cloud Run functions, App Engine | Kubernetes, VMs, Docker Compose | See Hosting below |
| worker | Lambda (SQS trigger), ECS tasks, AWS Batch | Functions (queue trigger), Container Apps jobs, WebJobs | Cloud Run jobs, Cloud Run functions | Kubernetes Jobs, Celery, Sidekiq, Hangfire | Functions cap run time (Lambda at 15 minutes); long jobs belong in container jobs or batch |
| scheduler | EventBridge Scheduler | Functions timer trigger, Container Apps jobs (cron), Logic Apps | Cloud Scheduler | cron, Quartz, Hangfire | Cron is not a per-entity delay. "Remind in 24 h" is a delayed message or one-time schedule |
| workflow | Step Functions | Durable Functions, Logic Apps | Workflows | Temporal, Camunda, Airflow | Step Functions and Workflows define steps as JSON/YAML; Durable Functions and Temporal in code. Airflow (MWAA, Cloud Composer) is for data pipelines, not request sagas |
| sql-db | RDS (PostgreSQL, MySQL, SQL Server), Aurora | Azure SQL Database, Azure Database for PostgreSQL, SQL Managed Instance | Cloud SQL, AlloyDB, Spanner | PostgreSQL, MySQL, SQL Server | Spanner scales writes horizontally with strong consistency (Aurora PostgreSQL Limitless Database shards writes too); the rest scale up plus read replicas |
| nosql-db | DynamoDB, DocumentDB | Cosmos DB, Table Storage | Firestore, Bigtable | MongoDB, Cassandra | The partition key is the design decision on every one of them |
| cache | ElastiCache, MemoryDB | Azure Managed Redis | Memorystore | Redis, Valkey, Memcached | MemoryDB is durable enough to be a primary store. Azure Managed Redis replaces Azure Cache for Redis |
| object-storage | S3 | Blob Storage | Cloud Storage | MinIO, Cloudflare R2 | Storage classes differ in whether cold data is online (see Data). R2 charges no egress |
| search | OpenSearch Service | AI Search | Agent Search (formerly Vertex AI Search) | Elasticsearch, OpenSearch, Meilisearch | OpenSearch and Elasticsearch give you index control to tune; Agent Search is more of a finished product. Elastic Cloud runs on all three |
| analytics | Redshift, Athena | Fabric, Data Explorer, Synapse | BigQuery | Snowflake, ClickHouse, Databricks | Athena and BigQuery (on-demand) bill per data scanned. Snowflake and Databricks run on all three clouds |
| queue | SQS | Service Bus queues, Storage Queues | Cloud Tasks, Pub/Sub | RabbitMQ, ActiveMQ | See Messaging below |
| stream | Kinesis Data Streams, MSK (Kafka), SNS, EventBridge | Event Hubs, Event Grid, Service Bus topics | Pub/Sub, Managed Service for Apache Kafka | Kafka, NATS, Redpanda | Two different shapes hide here: a replayable log and push pub/sub |
| auth | Cognito, IAM Identity Center | Entra ID, Entra External ID | Identity Platform, Firebase Authentication | Auth0, Keycloak, Okta | Staff and customers are different products. Service identity is IAM roles / managed identities / service accounts |
| secrets | Secrets Manager, KMS | Key Vault | Secret Manager, Cloud KMS | HashiCorp Vault | KMS-style services hold encryption keys; secret stores hold values your app reads |
| waf | AWS WAF, Shield | Front Door WAF, DDoS Protection | Cloud Armor | Cloudflare WAF, ModSecurity | A WAF attaches to the edge or load balancer, so it follows your CDN and LB choice |
| external | Amazon SES (email) | Azure Communication Services (email, SMS) | none first-party for email; use a third party | Stripe, Twilio, SendGrid | Wrap every third party with timeouts, retries and a circuit breaker |
| monitoring | CloudWatch, X-Ray | Azure Monitor, Application Insights | Cloud Monitoring, Cloud Logging, Cloud Trace | OpenTelemetry, Prometheus + Grafana, Datadog | Instrument with OpenTelemetry and the backend stays swappable |
Hosting
| AWS | Azure | Google Cloud | |
|---|---|---|---|
| Virtual machines | EC2 | Virtual Machines | Compute Engine |
| Managed Kubernetes | EKS | AKS | GKE |
| Serverless containers | ECS on Fargate | Container Apps | Cloud Run |
| Functions | Lambda | Functions | Cloud Run functions (formerly Cloud Functions) |
| Web app PaaS | Elastic Beanstalk | App Service | App Engine |
| Static SPA hosting | Amplify Hosting | Static Web Apps | Firebase Hosting |
- Default to serverless containers for new stateless services: low ops, autoscaling, and a container image that runs anywhere.
- Kubernetes when you need the Kubernetes API itself (many teams on one platform, a service mesh, GPUs, custom operators), not "because it scales better". You take on upgrades, node pools and networking.
- Functions for event handlers, webhooks and timers. Watch run-time caps and cold starts. The fixes are a warm minimum: provisioned concurrency (Lambda), always-ready instances on Flex Consumption (Azure Functions), min instances (Cloud Run).
- VMs for legacy software, special OS needs or licensing. You patch them.
- Scale to zero suits dev and spiky traffic: Cloud Run, Container Apps and functions do it; a warm minimum costs money but removes cold starts.
Edge and API management
The chain is the same everywhere: DNS → CDN + WAF at the edge → global or regional load balancer → API gateway → services on a private network.
AWS: Route 53 → CloudFront + AWS WAF → ALB or API Gateway → ECS on Fargate
Azure: Azure DNS → Front Door (WAF) → API Management → Container Apps (internal)
Google: Cloud DNS → global Application LB + Cloud CDN + Cloud Armor → Cloud Run
- A global L7 edge gives TLS close to users, caching and failover between regions: CloudFront (AWS; DNS steers users to the nearest edge, and origin failover triggers on errors or timeouts), Front Door (Azure) and the global external Application Load Balancer (Google Cloud), both anycast with health probes.
- DNS-based routing (Route 53 policies, Traffic Manager, Cloud DNS routing policies) is simpler but fails over only as fast as clients drop cached records.
- The API gateway validates tokens, applies rate limits per key, and versions the API. Keep business logic out of it.
- Azure notes: Application Gateway is a regional L7 balancer inside your VNet, often unneeded when Front Door reaches origins over Private Link. AWS notes: API Gateway has REST and HTTP API types with different features and prices. Google Cloud notes: Apigee is the full API management product; API Gateway is the lighter one.
Identity and secrets
Answer three questions, in this order:
- Who signs in? Staff (workforce identity: IAM Identity Center, Entra ID, Cloud Identity or Google Workspace, Okta) or customers (customer identity: Cognito user pools, Entra External ID, Identity Platform or Firebase Authentication, Auth0, Keycloak). SPAs use the authorization code flow with PKCE, and the API validates the token at the gateway and again in the service.
- How do services authenticate to each other and to cloud services? With a platform identity, so there's no secret to rotate: IAM roles (task, function or instance roles), Azure managed identities, Google service accounts attached to the workload. CI pipelines use OIDC federation (GitHub Actions to an IAM role, Azure workload identity federation, Google Workload Identity Federation).
- Where do the remaining secrets live? Third-party API keys and certificates go in a secrets store (Secrets Manager, Key Vault, Secret Manager, Vault), read at runtime with the service's own identity, cached in memory and never baked into images or repos.
- AWS notes: Secrets Manager can rotate database credentials itself; KMS keys never leave KMS.
- Azure notes: Azure AD B2C is closed to new customers, so new customer-facing apps use Entra External ID. Key Vault holds secrets, keys and certificates, and throttles, so cache reads.
- Google Cloud notes: Secret Manager versions every secret; rotation schedules notify you through Pub/Sub and your code does the rotating.
Data
- Relational (orders, money, inventory, reporting): RDS or Aurora, Azure SQL or Azure Database for PostgreSQL, Cloud SQL or AlloyDB. Spanner when one relational database must scale writes horizontally or span regions with strong consistency. A managed PostgreSQL keeps lock-in low: the engine is the same everywhere.
- NoSQL (massive key-based access, flexible schema): every one of them limits throughput per partition, so a hot key gets throttled while the table looks idle. DynamoDB: about 3,000 reads and 1,000 writes (capacity units) per second per partition. Cosmos DB: 10,000 RU/s per physical partition and 20 GB per logical partition. Firestore: no fixed number, but a single document can't take unlimited writes, so shard hot counters. Bigtable: sequential row keys pile onto one node. Choose a key with high cardinality that appears in most queries.
- Cache: managed Redis-compatible everywhere (ElastiCache, Azure Managed Redis, Memorystore). The patterns from the Caching lesson don't change.
- Object storage classes trade storage price against access cost and minimum duration:
| Online, infrequent | Coldest class | Is the coldest class online? | |
|---|---|---|---|
| AWS S3 | Standard-IA, Glacier Instant Retrieval | Glacier Deep Archive | No: restore first, hours |
| Azure Blob | Cool, Cold | Archive | No: rehydrate first, hours |
| Google Cloud Storage | Nearline, Coldline | Archive | Yes: milliseconds, higher read cost |
Lifecycle rules move objects between classes automatically; early deletion is charged against each class's minimum duration. Browsers upload directly with short-lived signed URLs (S3 presigned URLs, Azure SAS, Cloud Storage signed URLs).
Messaging
| Shape | What it is for | AWS | Azure | Google Cloud | Other |
|---|---|---|---|---|---|
| Queue | Commands and work items; each message goes to one consumer, retries, dead-letter queue | SQS (FIFO for ordering) | Service Bus queues (sessions for ordering) | Cloud Tasks, Pub/Sub pull subscriptions | RabbitMQ |
| Push pub/sub | "Something happened, react if you care"; fan-out to handlers | SNS, EventBridge | Event Grid, Service Bus topics | Pub/Sub | NATS |
| Stream (log) | High volume, ordered per partition, replay, many independent readers | Kinesis Data Streams, MSK | Event Hubs | Pub/Sub (with retention and seek), Managed Service for Apache Kafka | Kafka, Redpanda |
- A queue tracks each message (lock, delivery count, dead-letter). A stream tracks one position per partition: a poison event either blocks the partition or gets skipped, so catch failures per event and park them somewhere durable.
- Brokers cap message size. Send big payloads as an object-storage reference (the claim check pattern).
- Azure notes: commands to Service Bus, facts to Event Grid, streams to Event Hubs (which also speaks the Kafka protocol). AWS notes: SNS fanning out to SQS queues is the classic pub/sub-to-queue pattern; EventBridge adds content-based routing. Google Cloud notes: Pub/Sub covers queue, pub/sub and much of the stream use case on its own, with ordering keys and dead-letter topics.
Observability
- Every cloud has metrics, logs, traces and alerts: CloudWatch (and X-Ray), Azure Monitor (Application Insights, Log Analytics), Cloud Monitoring, Cloud Logging and Cloud Trace.
- Instrument with OpenTelemetry, so you can switch backends or add Datadog or Grafana without touching code. Each cloud also offers managed Prometheus (Amazon Managed Service for Prometheus, Azure Monitor managed service for Prometheus, Google Cloud Managed Service for Prometheus), and AWS and Azure offer managed Grafana; on Google Cloud you run Grafana yourself or use Grafana Cloud.
- Propagate trace context through HTTP and through messages, or traces stop at the queue.
- Alert on what users feel (error rate, p95 latency, queue age, dead-letter count), not on CPU.
- Query languages differ: CloudWatch Logs Insights, KQL in Azure, the Logging query language (or SQL in Log Analytics) on Google Cloud.
Resilience
- Zones first: all three clouds split regions into availability zones. Spread instances across zones and use zone-redundant data services; on some services that is a creation-time choice.
- Regions second, only when the requirements justify the cost: Aurora Global Database, DynamoDB global tables, S3 replication; Azure SQL failover groups, Cosmos DB multi-region, geo-redundant storage; Cloud SQL cross-region replicas, Spanner multi-region, dual- and multi-region Cloud Storage buckets. Traffic moves with Route 53 failover records, Front Door priority routing, or Google's global load balancer.
- Check what a DR feature copies. Some copy only configuration, not data. Azure Service Bus Geo-DR, for example, copies entities but not messages.
- Azure notes: region pairs (West Europe and North Europe) drive geo-redundant storage replication and staggered platform updates; newer regions may have no pair.
- Write RPO and RTO per data store, script the failover, and rehearse it (AWS Fault Injection Service, Azure Chaos Studio, or open-source tools such as Chaos Mesh).
Cost gotchas
- Log ingestion is billed per GB on every cloud and can cost more than compute. Sample traces, drop noisy logs, use cheaper log tiers.
- Provisioned capacity bills while idle: database instances, DynamoDB provisioned capacity, Cosmos DB RU/s, Kinesis shards, Event Hubs units, Spanner nodes, API gateway units.
- Data transfer: internet egress everywhere, cross-region replication, and cross-zone traffic on some clouds (AWS and Google Cloud). On AWS, NAT gateways charge per GB processed.
- Pay-per-scan analytics: a
SELECT *over a large table in Athena or BigQuery on-demand bills for every byte read. Partition and cluster tables, select only the columns you need. - Per-environment fixed costs (private endpoints, NAT gateways, premium tiers) repeat in dev, test and staging.
- Levers: scale to zero in dev, commitments in production (Savings Plans and Reserved Instances, Azure reservations and savings plan, committed use discounts), and tags or labels with budgets per team and environment.
How to talk about it in a design review
- Generic first, product second: "A durable queue between checkout and payment. On AWS that's SQS FIFO with the order id as the message group; on Azure, Service Bus with sessions."
- Say why this provider: "We're on Google Cloud because the team runs GKE today and we have committed spend", not "because it's the best cloud".
- Name the tier and the limits that shape the design: per-partition throughput, message size caps, function time limits.
- Tell the identity story in one breath: "Customers sign in through the customer identity service, the gateway validates the token, services use platform identities, remaining secrets live in the secrets store and are cached."
- State resilience as numbers: "Zone-redundant in one region; failover to a second region with RPO in seconds and RTO under an hour."
- Name the top cost drivers, and acknowledge the alternative you rejected ("self-hosted Kafka would work; nobody wants to run it").
Practise this concept
In the app: a short video and a 12-question concept check. Free with a GitHub account.
Open in the app