Measuring a workload · Joshua Wong

Start with the guided lesson. These sections develop the mechanisms and tradeoffs beyond that first example.

A fast average can conceal an unusable service

A marketplace launches a limited collection at noon. Its catalog has a million products, but one product receives most purchase attempts. Before launch, the database sustained a high rate of independent updates. At noon, the application begins timing out although CPU usage remains below its nominal maximum. Adding application servers increases the number of waiting requests. Some customers retry several times. A dashboard reports an acceptable average latency because fast catalog reads greatly outnumber the purchases that matter.

This scenario contains several different problems. Work is unevenly distributed. Requests compete over the same fact. Offered traffic differs from successful business activity. A retry policy changes the workload while the service is already overloaded. Finally, the measurement combines operations with different importance. The first responsibility of an engine designer is to distinguish these problems before selecting a data structure or a distributed topology.

A workload is a description of what the system must do, not just how many clients connect. It includes operation types, data sizes, access distributions, dependencies between operations, transaction boundaries, correctness requirements, and how all of those change over time. A database can be excellent for one workload and unsuitable for another even when the two applications have the same number of users.

Transactions and analytics consume resources differently

Online transaction processing, usually abbreviated OLTP, concerns relatively short operations over current operational facts. A purchase checks stock, creates an order, and changes ownership. A bank transfer reads and updates a few balances. The business expects related changes to obey an invariant and complete within an interactive latency budget. Individual operations may be small while aggregate traffic is large.

Online analytical processing, or OLAP, examines substantial portions of a dataset to produce summaries. A quarterly revenue report may scan billions of records while returning only a few dozen groups. Its dominant costs can be decompression, scanning, aggregation, and transferring intermediate results. A point lookup index might provide little benefit when nearly every row contributes to the answer.

The distinction describes work, not an immutable product category. A relational engine can process transactions and reports. The important question is whether the work shares CPU, caches, storage bandwidth, and concurrency mechanisms. A long report may retain old row versions, consume memory needed by transactions, or displace a hot working set. An architecture described as hybrid transactional and analytical processing, or HTAP, must explain how it reconciles those competing demands. A separate analytical copy changes resource isolation and freshness together; it does not make the analytical work disappear.

Consider two illustrative requests. Request A reads one account through its primary key and changes a 200-byte balance record. Request B scans 400 GB of historical payments to group revenue by country. Giving both requests the same timeout, admission policy, and resource allocation is unlikely to serve either well. Their result sizes may be similar, but the work needed to produce those results is radically different.

Count the operation at every boundary

A user intention is not a transport attempt. A transport attempt is not an admitted transaction. An admitted transaction is not necessarily a successful commit. A commit can create multiple physical writes and many client deliveries. Keeping these counters distinct prevents an apparently impressive throughput number from hiding wasted work or missing outcomes.

business intention
    -> one or more request attempts
        -> admitted execution
            -> committed or aborted transaction
                -> storage work
                -> zero or more delivered updates

The diagram is an accounting model. It does not require one separate software component per arrow. A single integrated engine can perform several steps while still needing separate measurements for them.

Suppose, illustratively, that 10,000 buyers each initiate one purchase. Network timeouts cause 4,000 additional attempts. The server therefore receives 14,000 attempts for 10,000 business intentions. If only 2,000 items exist, at most 2,000 purchases should succeed. The remaining intentions may correctly return sold out. Reporting only 14,000 requests processed says little about success, fairness, or correctness. Reporting only 2,000 successful purchases hides whether rejected buyers waited 20 milliseconds or 20 seconds.

The same accounting applies below the transaction layer. One transaction that updates a table and three secondary indexes creates more maintenance than one table mutation suggests. Three replicated copies multiply some storage and network work. Broadcasting a committed change to hundreds of subscribers creates another workload altogether. Scaling decisions require knowing which multiplier is currently dominant.

Workload envelopes and budgets

A useful workload specification describes a normal envelope and exceptional conditions. For example, a service may expect 20,000 transactions per second during normal traffic, 80,000 for ten seconds after a promotion, and a tenfold skew increase on the hottest key. It might require a 50-millisecond p99 latency for admitted regional purchases and permit analytical copies to lag by 30 seconds. These values are illustrative targets, not product capabilities.

Each target must identify the observation point. Does latency start when the user clicks, when the request reaches a gateway, or when a worker begins execution? Does completion mean an in-memory mutation, a durable commit, or delivery to a second device? A measurement can be internally correct while answering the wrong product question.

A resource budget expresses work in units that can be composed. An operation may require 30 microseconds of CPU time, 500 bytes of log output, and 2 KB of subscriber payload. At 20,000 operations per second, those assumptions imply 0.6 CPU-seconds per second, 10 MB per second of log bytes, and 40 MB per second of subscriber payload before replication and protocol overhead. The calculation is useful because it identifies candidate ceilings, not because it predicts exact production performance.

Latency budgets describe dependencies. Four serial network exchanges cannot be compressed by merely provisioning more connections for that same operation. Independent operations can overlap while the one request still waits for its own dependency chain. This difference explains why high aggregate throughput can coexist with disappointing individual response time.

Model the mix rather than one representative request

Assume a constructed normal workload has three operation classes: 70% item lookups, 20% inventory changes, and 10% history summaries. Their average CPU demands are respectively 10, 50, and 200 microseconds. The weighted demand is 0.70 × 10 + 0.20 × 50 + 0.10 × 200 = 37 microseconds per operation. At 20,000 operations per second, the workload consumes an illustrative 0.74 CPU-seconds per second before coordination, background work, and operating-system overhead.

That average is useful for resource accounting but insufficient for scheduling. The summary query is twenty times as expensive as the lookup, and its service-time variation can be much larger. A burst of summaries can delay inventory changes if they share one execution queue. The same long-run percentages can produce different latency when arrivals are smoothly interleaved, clustered into bursts, or triggered simultaneously by a report scheduler. A workload specification therefore needs temporal shape as well as aggregate proportions.

Data dependence creates another difference. Twenty thousand independent ownership changes can potentially spread across partitions. Twenty thousand changes to one auction cannot use that independence while preserving the auction’s ordering rule. Two workloads with identical statement text and row counts can behave differently because their parameter values select different keys. Benchmark generators should model that parameter distribution instead of sampling uniformly by default.

A useful resource model records a vector for each class: CPU time, bytes read, bytes written, log bytes, network bytes, and protected-resource occupancy. Multiplying those vectors by class rates reveals candidate bottlenecks. It also prevents a misleading optimization: reducing CPU on lookups may barely help when the sale is limited by one inventory decision or by subscriber delivery. The next resource ceiling is often different from the current one.

Concurrency requirements need an arrival model

“Support 100,000 users” omits how often they act. If each connected user submits one action every ten seconds, the business arrival rate is 10,000 actions per second. If each action changes data visible to twenty clients, delivery demand can reach 200,000 client updates per second. If users merely retain an idle connection, memory and connection management may dominate while transaction execution remains quiet.

Now suppose a major announcement makes all users act within one second. The average over an hour can remain low while a one-second burst exceeds sustainable capacity by an order of magnitude. A system needs an explicit policy: absorb the burst within a bounded queue, reject some attempts, reserve units through a waiting room, or relax the requested latency. None of these choices is visible in a count of registered accounts.

An arrival model should also state whether later requests depend on earlier responses. A game client might cap itself at one outstanding authoritative action. A backend job can produce thousands of independent updates regardless of response latency. The former naturally reduces offered load when the server slows; the latter can keep growing the backlog. Using one model to predict the other gives an inaccurate picture of overload behavior.