Find the resource that fills up first

The engine’s CPU is only half busy, yet requests are slowing down. More processor cores might help, but only if processor time is the limiting resource. Storage bandwidth, memory, or one contended record may be exhausted first.

A resource demand is the amount of a resource consumed by one operation. We will use fixed illustrative demands to calculate ceilings, then explain why those ceilings are not operating targets.

Keep units attached to the calculation

Assume two cores provide at most two CPU-seconds of processing per second. Each request uses 2 ms, or 0.002 CPU-seconds. The CPU-only ceiling is 2 / 0.002 = 1,000 requests per second.

The same request causes 0.1 MB of storage traffic, including the maintenance work counted by this model. If usable bandwidth is 50 MB/s, the storage ceiling is 50 / 0.1 = 500 requests per second.

CPU budget
2 CPU-seconds each second
0.002 per request
Ceiling: 1,000 requests/s

Storage budget
50 MB each second
0.1 MB per request
Ceiling: 500 requests/s

Real requests have different demands. A large scan and a small update cannot be counted as interchangeable operations. Maintenance such as compaction and backup also consumes resources. Omitting it makes the predicted sustainable rate too optimistic.

Leave capacity for variation and recovery

A mathematical ceiling assumes every stated resource is fully usable with no harmful interference. Real services need spare capacity, or headroom, for bursts, variability, background tasks, and failure recovery. At a ceiling, even a small surplus can create a growing queue.

Memory is a different kind of budget. A request may hold buffers or transaction state while waiting. More concurrent work can therefore consume memory even when CPU activity is low. Limiting concurrency can protect that budget.

Share capacity between customers deliberately

A tenant is a customer or workload sharing the service. If one tenant launches an expensive scan, it may evict useful pages or occupy storage bandwidth needed by others. Per-tenant admission, memory accounting, and scheduling can limit that interference.

A single global request limit does not guarantee fairness when requests have very different costs. Estimate cost where useful, then measure actual resource use and enforce limits at the resource boundary.

Double only the CPU

We add two more cores while storage stays at 50 MB/s. What happens to the two ceilings?

Recalculate each resource separately

The CPU ceiling becomes 4 / 0.002 = 2,000 requests/s. The storage ceiling remains 500. The combined model still limits this workload to at most 500, before allowing headroom. A CPU purchase does not solve a storage bottleneck.

Capacity planning starts with a workload and its demands, then checks each resource and failure state. It is not a conversion from connection count to machine size.

Google’s load-balancing discussion considers load and resource limits. More detailed accounting examples remain in the extended reference.

Extended reference

The extended chapter preserves the original technical coverage, additional examples, and citations. The explanation above supplies the concepts needed for this page; the reference develops the specialist cases.

Find a section from the original chapter

• Learning outcomes
• A spare CPU does not imply spare database capacity
• Per-operation demand
• Memory as capacity and retention
• Storage bandwidth and maintenance debt
• Tenant isolation
• Hot keys and uneven work
• Autoscaling and useful capacity
• Planning for a failure state
• Constructing a resource-demand matrix
• Capacity coefficients are distributions
• Headroom must cover variance and transitions
• Resource ownership makes memory diagnosable
• Admission can use estimated and actual cost
• Maintenance is a producer-consumer system
• Tenant placement and elephant tenants
• Autoscaling has a time-to-usefulness curve
• A comparative operational scenario
• Capacity as an explicit promise
• A resource-budget table makes assumptions reviewable
• Case connection and design conclusion
• Sources and further reading