Hardware and operating-system costs — extended reference

This reference preserves the earlier detailed chapter and its citations. Start with the guided explanation, then use these sections for additional mechanisms and qualifications. References to the original lesson sequence are historical.

Part 1: Foundations: understanding work, time, and resources

Prerequisites: Lesson 01 and the preceding foundations.

Learning outcomes

• Trace a request through CPU caches, memory, the operating system, and persistent storage.
• Calculate copying and storage costs using consistent byte units and distinguish elapsed time from CPU occupancy.
• Identify which failure boundary a memory write, filesystem write, and durable flush cross.

A database request is a path through hardware

A lookup appears simple in application code: find a key and return a value. At the machine level, it may traverse several pointers, translate virtual addresses, bring cache lines into a core, wait for another core’s ownership, copy bytes into an output buffer, enter the kernel, and enqueue network transmission. When the key is absent from memory, storage and I/O scheduling add further work. The cost of the lookup is the sum of these activities along its dependency path, with some work overlapped and some serialized.

The motivating scenario is a database with enough RAM to hold its dataset that nevertheless has high latency. One engineer proposes more memory. Another proposes more cores. Neither change addresses the problem if execution is dominated by dependent cache misses or contention over a shared cache line. The useful investigation starts with the physical access pattern rather than the dataset’s nominal size.

This lesson introduces the hardware and operating-system vocabulary needed to interpret that investigation. Any example timings are illustrative; actual times depend on processors, devices, topology, operating systems, and load. Relative cost models are more portable than memorized latency tables.

Registers, caches, and dependent memory access

A processor can execute many instructions while waiting for some operations, but it cannot execute an instruction whose input is not yet available. A chain of dependent pointers exposes this limitation. If finding the next node requires reading the current node, the processor cannot request every future node at once. Several individually small delays can therefore add directly to lookup latency.

CPU caches retain recently used memory in units called cache lines. A cache hit avoids a more distant access. Sequentially scanning a compact array often lets hardware fetch future lines ahead of use. Traversing scattered heap objects may prevent effective prefetching and use only a few useful bytes from each fetched line. Two data structures with the same asymptotic lookup complexity can differ substantially in the amount of physical data they move.

Consider an illustrative B+ tree lookup that follows four dependent nodes. If all nodes are already in nearby caches, execution can be short. If each node requires a 100-nanosecond memory access, the dependent chain alone contributes about 400 nanoseconds. That is not a measured B+ tree latency and excludes comparisons, branches, address translation, and synchronization. It demonstrates why locality affects the constant factors hidden by big-O notation.

A cache-conscious design groups frequently accessed metadata together and avoids loading unrelated fields. This can conflict with other goals. Packing mutable fields from different workers into one line may create false sharing. Separating every field can waste space and increase the number of lines fetched for a common operation. The correct layout depends on which fields are read and written together, and by whom.

Virtual memory and address translation

Applications generally use virtual addresses. Hardware and the operating system translate these to physical memory mappings. A translation lookaside buffer, or TLB, caches address translations. A TLB miss may require additional memory accesses to walk page tables. Large working sets with poor locality can stress both data caches and translation caches.

A virtual-memory page and a database page are different concepts. The former is an operating-system and hardware mapping unit; the latter is an engine’s storage and buffer-management unit. They may have related sizes but do not share ownership semantics automatically. A database pin prevents its own buffer manager from reusing a frame. It does not by itself prevent operating-system paging unless the engine separately uses appropriate facilities and privileges.

Allocating memory does not guarantee that every byte has already been physically populated. First touch can allocate or map pages, and page faults can introduce variable delay. Benchmark warm-up must therefore distinguish loading executable code, populating database caches, establishing virtual-memory mappings, and reaching storage steady state. A warm run answers a different question from the first request after failover.

Huge pages can reduce translation overhead for suitable workloads, but they can also affect allocation, compaction, and pause behavior. They are a tuning choice with platform-specific consequences. A lesson about databases should not convert a generally useful hardware feature into a universal recommendation without measuring the actual engine and operating-system interaction.

NUMA and the location of memory

On a nonuniform memory access machine, abbreviated NUMA, processors have different access costs to different memory regions. Memory attached near one socket can be more expensive for a core on another socket to access. A database can therefore have unused memory bandwidth on one node while workers repeatedly access remote allocations on another.

First-touch allocation, thread placement, and ownership of partitions can affect locality. If one initialization thread touches every buffer page, placement may differ from the desired runtime distribution. If workers migrate frequently, a layout that was initially local can become remote. These are examples of how scheduling and allocation policies interact with database partitioning.

Suppose two workers process independent tenant partitions. Assigning each partition’s frequently used data to memory near its worker may improve locality. If a global transaction then accesses both partitions, it crosses those ownership boundaries. NUMA-aware partitioning benefits the common local operation while adding complexity to rebalancing and cross-partition work. It is the same shape of tradeoff that appears later across separate machines, on a different cost scale.

Storage devices are queues and failure domains

A storage device is not just a fixed latency number. It has request queues, internal parallelism, bandwidth limits, firmware behavior, and background work. Sequential bandwidth and random I/O operations per second describe different access patterns. A benchmark reading large contiguous blocks does not predict the latency of small random writes under simultaneous compaction.

Queue depth is the number of outstanding I/O requests. Increasing it can improve device utilization up to a point. Beyond that point, additional requests mostly wait. The resulting latency can harm interactive transactions even while aggregate throughput remains high. A database needs to coordinate background and foreground I/O rather than allowing every subsystem to saturate the device independently.

Flash storage introduces erase and write-management behavior beneath the database’s view. The engine’s compaction and checkpoint patterns interact with the device’s own amplification and garbage collection. Logical bytes written by transactions, host bytes written to the device, and internal flash bytes are distinct quantities. A realistic capacity plan keeps the relevant layers separate.

Failure assumptions also matter. A write acknowledged by an interface may be in a volatile buffer. Durable behavior depends on the operating-system call, filesystem, device, configuration, and whether the hardware honors the requested ordering and persistence. An engine cannot establish power-loss durability merely by copying bytes into an application buffer.

Buffered I/O and the page cache

With buffered file I/O, the operating system can retain file contents in its page cache. A read may be served from memory even though the application called a file API. A write may return after bytes have been copied into kernel-managed memory, with device persistence deferred. The Linux write interface explicitly distinguishes successful writing from a guarantee that data has reached durable storage.

A database may also maintain its own buffer pool. This creates two caching layers with different information. The database understands page types, transaction visibility, and query access. The operating system sees file offsets and memory pressure across processes. Duplicate caching can waste memory, but the operating system’s cache can also provide useful reuse or simplify implementation. The choice is an engineering decision rather than an automatic mistake.

Direct I/O can bypass aspects of buffered caching, subject to platform and filesystem requirements. It can give the engine more explicit control over memory and I/O, while introducing alignment, scheduling, and correctness obligations. Memory mapping exposes file contents through virtual addresses, shifting some page-in and page-out activity into fault handling. It changes the interface to storage but does not make persistence ordering or I/O failures cease to exist.

System calls, copying, and batching

A system call crosses from application execution into the kernel. The transition has a cost, but the work performed within the call often matters more than the transition alone. A write can copy data, allocate structures, wait for space, or interact with filesystem metadata. A socket operation can enqueue bytes without transmitting them immediately.

Batching reduces repeated per-call overhead. Sending a collection of small results together can avoid many transitions and headers. It may also increase the delay before the first result leaves. An engine that batches indefinitely can have excellent amortized efficiency and unacceptable interactive latency. Batch policy must include a size or time bound matched to the workload.

For an illustrative 4 MiB result copied three times, the copies move 12 MiB of payload. This figure excludes reading the original data, cache-line ownership, metadata, and transmission. At a sustained effective copy rate of 12 GiB/s, those copies alone take about 0.977 milliseconds: 12 MiB / 12 GiB/s = 1/1024 s. The calculation uses binary units consistently. It is a cost illustration, not a claim about a particular machine.

Avoiding copies can reduce this work, but a zero-copy path must preserve buffer lifetime and isolation. If the engine exposes a page directly while another worker mutates or reuses it, the saved copy can become corruption or data leakage. Copying often provides ownership separation. Removing it transfers that responsibility to reference counting, pinning, immutability, or another explicit lifetime mechanism.

Persistence ordering and acknowledgment

A write-ahead log separates recording a change from installing every changed database page. The crucial ordering is that log information needed for recovery becomes durable before a dependent page is allowed to become durable in a way recovery cannot explain. A transaction may then acknowledge after its required commit information is durable, while page writeback happens later.

This structure can turn many random page flushes into a sequential log append and shared synchronization. PostgreSQL’s WAL documentation describes this separation. The general design consequence is that the normal transaction path and the recovery path are deliberately coupled: the transaction avoids some immediate page work because recovery knows how to reconstruct it.

The operating system may coalesce writes, and a database can group multiple transactions into one synchronization operation. These optimizations change scheduling and amortization. They do not eliminate the need to identify which transaction’s durable boundary has been reached. An acknowledgment that overtakes its required durable watermark changes the contract.

A worked access-path comparison

Consider two illustrative implementations of a frequently read account record. Design A stores the hot fields in a compact array indexed through a dense identifier. Design B stores each field in a separate object reached through pointers. Both can retrieve the same logical information. Design A might touch one or two nearby lines; Design B may perform several dependent misses and allocations.

The compact design benefits repeated access and predictable layout. It can suffer when identifiers are sparse, records vary greatly in size, or independently updated accounts share cache lines. The pointer-rich design can make flexible ownership and variable-sized data simpler, while spending more memory and instruction work. There is no architecture-independent winner. The workload contract from Lesson 1 tells us which access pattern to optimize.

Now add persistent writes. If the account update generates 300 bytes of log data, a high rate of updates can make log bandwidth or synchronization the dominant cost even after lookup becomes very fast. A faster in-memory structure improves one stage while leaving a different ceiling untouched. This is why optimization must trace the whole path and measure the resource that is actually constrained.

Count bytes moved, not just bytes requested

Consider a constructed service returning a 4 MiB result. One stage copies records into an encoded message, another copies the message into a transport buffer, and a third copies it into another owned buffer. Those three copies account for 12 MiB of copied payload. A copy reads and writes memory, so the corresponding memory traffic can exceed 12 MiB before cache behavior and device transfers are considered. The useful application data remains only 4 MiB.

At 500 such results per second, three payload copies represent 6,000 MiB per second of copied data. This is a dimensional calculation, not a bandwidth measurement. It explains why an application with a low transaction count can still stress memory bandwidth and CPU. Compression can reduce transmitted bytes while adding instructions and temporary buffers. Whether it improves latency depends on which resource was scarce, payload compressibility, and the amount of work that can overlap.

Scatter/gather interfaces allow a message to reference several existing buffers rather than first concatenating them. The benefit requires a valid buffer lifetime: bytes cannot be reused while the operating system or another asynchronous stage still needs them. “Zero copy” is therefore a claim about a specific boundary, not an assertion that no component copies or reads the data anywhere. Kernel, device, encryption, and user-space transformations must each be accounted for.

Storage throughput and storage latency answer different questions

A device can sustain high aggregate throughput by serving many requests concurrently while each request still has a substantial completion time. Suppose an illustrative device can complete 100,000 independent reads per second, and average read latency at a chosen operating point is 200 microseconds. Little’s Law gives an average of twenty reads in flight: 100000/s × 0.0002 s = 20. A client issuing one dependent read at a time cannot automatically obtain that aggregate rate.

An index lookup that reads a root, then an internal page, then a leaf may have dependencies that prevent issuing all three absent pages together. Independent queries can overlap their I/O, raising throughput, but one query still waits for its chain. Caching upper tree levels removes some of those dependencies. Prefetching helps when future addresses can be predicted. Neither technique should be assumed to remove every random miss.

Queue depth increases utilization until another limit appears. Beyond that point, additional queued requests mostly increase waiting. Device bandwidth numbers obtained with deep parallel queues therefore should not be inserted directly into a point-lookup latency estimate. Sequential bandwidth also differs from random small-read capacity. A workload model needs request sizes, access pattern, queue depth, and read/write mixture.

Solid-state storage adds internal behavior such as erase-block management and garbage collection. The database generally sees latency variation rather than the device’s complete internal schedule. Sustained measurements should therefore include a realistic occupancy and write history. A brief test on an empty device can omit steady-state work that a full, frequently rewritten device must perform.

A byte’s journey through a buffered write

The following trace is a simplified operating-system model, not a complete commit algorithm:

engine modifies private or cached state
    -> engine forms log bytes
    -> write submits bytes to kernel-managed state
    -> synchronization requests persistence
    -> storage reports completion under its guarantees
    -> engine advances its durable boundary

The caption for this trace is a sequence of ownership and acknowledgment boundaries. Completion of an earlier arrow does not imply completion of a later one. In particular, the Linux write interface can return after accepting bytes without guaranteeing persistence, and writes can be partial. Callers must handle the actual returned byte count and errors. Linux write semantics

A write loop also needs to distinguish interrupted progress from a nonretryable storage failure. Blindly retrying after an error without knowing which bytes were accepted can corrupt an append protocol unless record framing and offsets make retry safe. A log typically uses lengths, sequence information, and integrity checks so recovery can identify complete valid records. Lesson 24 explores how those records support a durable acknowledgment decision.

Synchronization is likewise not a magic word independent of the storage stack. Filesystem behavior, mount configuration, device caches, virtualization, and the stated failure model all affect what survives. A design should define whether it protects against process crashes, operating-system crashes, power loss, or loss of an entire machine. These events cross different boundaries and may require different mechanisms.

Hardware counters need a causal interpretation

High CPU utilization means cores are occupied under the measurement’s accounting rules; it does not mean they perform useful database work efficiently. They may spend time spinning on a latch, processing failed optimistic attempts, copying bytes, or waiting on dependent memory. Low CPU utilization can coexist with a saturated single worker, storage queue, or network path. An average over many cores can conceal one fully used execution resource.

A performance investigation should connect observations. A high last-level-cache miss rate together with scattered pointer accesses suggests a locality problem. A high context-switch rate together with lock wait time suggests scheduling and synchronization overhead. Increasing storage completion latency while queue depth grows suggests a device-side bottleneck. Each is a hypothesis supported by several measurements, not a diagnosis based on one counter.

Sampling profilers identify where execution time appears, but blocked time can need different instrumentation. If a request spends 40 milliseconds waiting for a durable log flush, a CPU profile may contain little evidence of those 40 milliseconds. Trace spans, lock waits, I/O completion measurements, and scheduler observations complete the picture. Timing from request admission to completion must include periods in which no application instruction runs.

Scenario: the larger machine is slower

Imagine a partitioned in-memory engine on one socket. Each worker owns a shard and accesses mostly local memory. After migrating to a larger two-socket host, initialization allocates most memory near socket A while half the workers run on socket B. Cross-socket traffic now affects those workers, and a shared statistics counter bounces between cores. The new machine has more total RAM and cores while the critical path becomes longer.

Three possible changes address different causes. Distributing allocation according to runtime ownership can improve memory locality. Partitioning or batching statistics updates can reduce cache-line sharing. Pinning selected workers can stabilize locality, though it can also reduce scheduling flexibility. None should be prescribed solely from the scenario; measurements should establish which effect dominates and whether the change harms load balance.

The broader lesson is that scale-up changes topology as well as quantity. More resources can expose a new communication boundary inside one computer. A database’s logical partitions, worker ownership, buffer allocation, and background tasks should be evaluated against that topology. The same principle later reappears across machines, where network distance makes the locality boundary more explicit.

Keep the hierarchy visible in the trace

CPU registers and nearby caches       instruction operands
          |
shared caches and local RAM           cache-line movement
          |
remote NUMA memory                    interconnect plus memory access
          |
engine buffers / operating-system cache
          |
persistent storage                   device queues and durability rules

This hierarchy describes categories, not fixed measured timings. A lookup can stop at several layers, while a write can become visible at one layer before reaching a durable boundary below it. Moving data upward can accelerate reuse; making it durable requires the appropriate downward protocol. A CPU profile mainly observes instruction activity and cannot independently identify every wait along that path.

Case interpretation and the remaining responsibility

The documented Linux distinction between writing and persistence, together with PostgreSQL’s WAL ordering, supports a useful architectural conclusion: “data is in memory” and “data is durable” answer different questions. An in-memory engine can be durable through a log. A disk-oriented engine can serve many reads from memory. Neither label alone establishes acknowledgment latency, recovery time, or loss behavior.

An engine designer must identify where bytes live, which component owns them, which operations copy them, and which failures can erase them. Those questions apply to buffers, caches, log records, network frames, and replica state. The local concurrency chapter follows execution resources and scheduling, because fast memory access helps only when useful work is allowed to run.

Sources and further reading

• Linux write system call
• PostgreSQL 18 WAL introduction

Course index · Lesson plan