Redis and memory-oriented designs — extended reference

This reference preserves the earlier detailed chapter and its citations. Start with the guided explanation, then use these sections for additional mechanisms and qualifications. References to the original lesson sequence are historical.

Part 8: Architectural case studies and synthesis

Prerequisites: Lesson 44 and the preceding foundations.

Learning outcomes

• Explain how in-memory representation and short execution paths can reduce ordinary request cost.
• Distinguish command execution from networking, persistence, replication, and background work.
• Evaluate memory capacity, hot-key occupancy, large commands, and acknowledgment contracts.

A fast cache becomes an accidental ledger

A game team introduces Redis to cache inventory summaries. A miss loads the summary from the durable ownership tables. Later, another team stores temporary trade reservations in the same service. Eventually an important purchase exists only as a Redis key with an expiration time. During memory pressure, eviction removes that key. The application interprets absence as evidence that the purchase never happened.

Nothing about fast memory access resolves this ambiguity. The application changed the meaning of the service without changing its contract. A disposable representation and an authoritative record need different behavior when memory fills, a machine restarts, or a replica becomes leader. This lesson uses memory-oriented systems to connect data structures, command execution, persistence, capacity, and application correctness.

The relevant question is not simply whether Redis is a cache or a database. It can be used in different roles. The engineer must specify which state is reconstructible, what an acknowledgment promises, which failures may lose data, and whether absence means “not cached,” “expired,” “deleted,” or “never existed.” Those distinctions belong in the application protocol.

Memory residency removes one cost category

A memory-resident working set avoids a storage read on ordinary accesses. The remaining path includes network handling, decoding, key hashing or comparison, object traversal, command execution, result construction, and delivery. Dynamic allocation, cache misses within RAM, scheduling, and contention can still dominate. Reading memory is not one uniform constant-time physical operation: contiguous bytes and pointer chains exercise hardware differently.

Consider a hypothetical session lookup. The application sends a 48-byte key and receives a 512-byte value. If all data is resident, no foreground device read is needed. But a remote call still pays connection processing and network delay. If the application accesses that session four times sequentially, dependent round trips can dominate the data lookup itself. Pipelining independent requests removes waiting between exchanges; it cannot remove dependencies that genuinely require earlier results.

Redis’s latency documentation explains that expensive commands, operating-system scheduling, persistence activity, and other environmental factors affect observed latency. Its command execution model should therefore be evaluated alongside command complexity and maintenance, rather than summarized as “RAM is fast.” Current product variants have differing threading and deployment features; inspect the selected version rather than assuming one thread describes every activity. Redis latency documentation

This same reasoning applies to SpacetimeDB and any memory-oriented engine. Eliminate a disk miss and the bottleneck moves toward computation, synchronization, network delivery, or persistence. The optimization changes the dominant cost rather than eliminating cost.

Native structures avoid unnecessary application work

A database exposing only opaque values may require the application to read, deserialize, change, serialize, and rewrite a large object. A service exposing operations on sets, hashes, sorted collections, and counters can perform a narrow change near the state. That reduces transferred bytes and can avoid application-level races.

Suppose a leaderboard contains one million scores. Reading the entire collection to add one score is wasteful. A sorted access structure can maintain order incrementally and answer a top-ranked range without returning all participants. Its insertion and retrieval costs depend on the representation and requested result size. Returning 500,000 entries still produces a large response even if locating the first entry is efficient.

A useful mental model separates three costs: finding the relevant structure, modifying it, and returning the result. A constant-time key lookup does not make every operation on the associated value constant time. Scanning a collection, deleting many members, or generating a large result can occupy execution resources long enough to delay unrelated clients.

request arrives
  -> locate collection by key
  -> inspect and modify relevant members
  -> construct result
  -> enqueue response bytes
  -> connection drains independently

Figure: A simplified command path. Fast key lookup is only one stage, and a response queue can remain expensive after logical execution finishes.

Application design should use the narrowest operation that expresses the intended change. That advice is about reducing work, not packing arbitrary business logic into an unbounded server-side command. A server-side function that loops through every account can remove round trips while creating a long scheduling obstacle.

Atomic execution has a defined boundary

If the server executes a single update atomically, other commands cannot observe its intermediate state under that command’s contract. This does not make a sequence of separate application requests one atomic business operation. Reading a balance and later setting a new value permits interference unless an appropriate conditional or transaction mechanism protects the sequence.

An illustrative compare-and-update protocol is:

repeat within bounded attempt budget:
    read current value and version
    compute proposed value without external side effects
    attempt update only if version is unchanged
    if success: return result
return conflict response

The annotation matters: external side effects must not occur on every retry. If computing the value charges a credit card, a conflict can duplicate the charge. A bounded attempt budget also prevents a hot key from creating unlimited retry work.

Alternatively, a short operation executed atomically near the state can check and update together. Which method is preferable depends on contention, execution duration, and supported semantics. Neither should be described as “no concurrency problem.” One changes waiting into retry risk; the other concentrates execution time inside an authoritative boundary.

A multi-key invariant adds placement concerns. If two account balances live on different shards, local atomic execution cannot by itself provide an atomic cross-shard transfer. Co-location can make the common transaction local, but may concentrate load. Splitting the invariant requires a distributed protocol or a different business contract.

Capacity starts with payload and grows outward

For an illustrative 100 million values averaging 64 payload bytes, payload alone is:

100,000,000 × 64 B = 6,400,000,000 B = 6.4 GB.

That excludes keys, collection metadata, object headers, alignment, allocator slack, indexes, client buffers, replication state, and persistence overhead. If keys average 24 bytes, they add another 2.4 GB before any structural overhead. Treating 6.4 GB as the required machine size would omit much of the representation.

A simple capacity model can use measured bytes per representative key class. If a realistic sample consumes 180 bytes per entry after overhead, 100 million entries require about 18 GB for those objects. This is a hypothetical measured coefficient, not a Redis benchmark. Different encodings and value distributions change it. Sampling must include small and large values, deletion patterns, and partially filled structures.

The machine also needs headroom. If the dataset uses 18 GB, response buffers use 1 GB, other engine state uses 1 GB, and a background process or workload burst needs 4 GB, a 20 GB limit cannot accommodate all phases. Operating close to the steady-state minimum can turn routine maintenance into an outage.

Memory is both a capacity and a latency resource. When the working set exceeds available physical memory, operating-system paging can introduce storage-scale delays. An application expecting memory-speed access may become unstable rather than gradually slower. Avoiding this regime requires accounting for process overhead and maintenance, not merely limiting logical payload.

A command lifecycle and memory budget

arrival -> queue -> command execution -> reply queued -> client receives
                       |                    |
                       +-> persistence      +-> replica progress

Figure: A conceptual lifecycle. Reply, persistence, and replication ordering depend on the configured contract; their lines must not be assumed equivalent.

Table: Payload arithmetic establishes a lower bound, while structural and transition costs determine the actual memory requirement.

Persistence describes restart outcomes

Redis documents snapshot persistence and append-only persistence, with configurable synchronization policies and the ability to combine mechanisms. Snapshots capture a point-in-time image; append-only persistence records operations for reconstruction. Their costs and failure windows differ. A policy that periodically synchronizes an append-only file does not imply every acknowledged command is already synchronized. Redis persistence documentation

The general recovery questions are independent of implementation branding. Is the durable representation a complete image, an operation log, or both? What establishes the authoritative end of the log? How does recovery recognize an incomplete final record? Does replay reapply logical commands or install physical state? How long can reconstruction take at the expected dataset size?

Imagine a hypothetical append-only deployment that acknowledges before a periodic durable flush. A power failure can remove acknowledged recent changes. Whether that is acceptable depends on the role. A cache can reload lost entries. A unique ownership ledger cannot silently reconstruct an acknowledged sale from an earlier snapshot and sell the item again.

Restart duration also matters. If reconstruction processes 50 GB at an effective 250 MB/s, the byte-processing component alone is 50,000 / 250 = 200 s, using decimal units. Allocation, index rebuilding, validation, and connection recovery can add time. A fast steady-state service can therefore have a slow recovery path.

Background persistence and copy-on-write

A snapshot procedure can preserve a consistent view while foreground activity continues by separating the background view from subsequently modified memory. In a copy-on-write organization, changes to shared pages create private copies. The mechanism reduces the need to stop ordinary work for the full image-writing duration, but changes memory pressure.

Consider a hypothetical 20 GB resident dataset whose background snapshot takes 60 seconds. During that interval, foreground updates dirty 3 GB of distinct previously shared memory pages. Copy-on-write can require roughly that additional page memory, with implementation and allocator qualifications. Repeated writes to the same already-copied page do not require copying the entire page again for that snapshot. Conversely, small updates scattered across many pages can duplicate far more memory than their changed payload bytes suggest.

This demonstrates why update locality matters. Updating one byte in each of one million distinct 4 KiB pages can touch approximately 4.096 GB of pages, even though changed payload is only 1 MB. The numbers are illustrative and omit page-sharing details, but the amplification principle is general.

A capacity plan must include the background task’s lifetime, update distribution, and storage speed. A slower snapshot lengthens the interval in which pages can diverge. Improving write bandwidth can therefore reduce both recovery-related I/O time and transient memory risk.

Expiration is a semantic decision

A time-to-live is useful for sessions, deduplication windows, temporary reservations, and cached results. It also establishes a boundary after which the system may forget information. If an operation identifier expires before a delayed client retry arrives, duplicate suppression can fail even if it worked perfectly while the record existed.

Suppose a purchase result is retained for 24 hours, but a mobile client can retry an offline request three days later. The result’s absence no longer distinguishes “never processed” from “processed and forgotten.” Solving that requires a longer-lived authoritative operation record, a protocol rejecting old attempts, or another reconciliation mechanism. Increasing cache capacity alone does not repair the semantic mismatch.

Time itself requires care. An expiration deadline based on wall-clock time can respond to clock adjustments differently from a monotonic elapsed-time budget. Distributed workers may disagree about current time. For a reservation that transfers ownership, server-side authoritative time and explicit state transitions are safer than trusting arbitrary client timestamps.

Expiration should not be confused with compensation. Removing an expired reservation key may release capacity only if the surrounding invariant is designed to recognize that transition. If reserved stock was separately decremented, deleting the key without restoring stock leaks inventory. The mechanism that forgets a key does not automatically reverse every business effect associated with it.

Eviction expresses a pressure policy

Redis offers configurable policies for choosing keys to evict when configured memory limits are reached, including policies restricted to expiring keys and policies that consider all keys. A no-eviction policy changes failure behavior toward rejecting applicable writes rather than selecting existing keys for removal. Redis eviction documentation

For a cache, eviction should lead to a controlled miss path. If every evicted key triggers an expensive database query, pressure can migrate into the authoritative backend. The cache has protected its own memory by increasing demand elsewhere. Admission, coalesced cache fills, and bounded fallback concurrency can prevent a miss storm from collapsing the origin.

For authoritative state, silently deleting arbitrary keys is generally incompatible with ownership or accounting invariants. Refusing writes is often more correct, although it creates availability consequences that the application must handle. A system can remain internally correct while being unable to accept more orders.

Mixing ephemeral and durable roles in one resource pool creates avoidable coupling. Separate deployments or clearly isolated policies can prevent a cache’s eviction behavior from becoming a ledger’s data-loss behavior. The right separation depends on operational costs, but the contract must remain explicit.

Replication and failover do not replace a commit contract

A replica can improve read availability and recovery speed while lagging the writer. If the writer fails after acknowledging a change but before the replica receives it, promoting that replica may lose the acknowledged change. Waiting for additional copies can reduce particular failure windows but must be understood within the actual replication and failover protocol.

Redis Cluster documents hash-slot placement, topology changes, and asynchronous replication with failure scenarios in which writes can be lost. Its mechanisms should not be equated with a general strongly consistent distributed transaction service. Redis Cluster specification

A generic failure trace makes the issue visible:

A is authoritative; B is a follower
A accepts operation X and replies success
X has not yet reached B
A becomes unavailable
B is promoted under the deployment's failover policy
client reads B and cannot find X

An application cannot resolve this simply by retrying a non-idempotent increment. It needs an operation identity and an outcome contract covering failover, or a stronger authoritative service. A cache can tolerate this history by rebuilding; a financial ledger usually requires different guarantees.

Clustering divides keys, not one indivisible hotspot

Hash-slot routing distributes keys across shard owners. A client may need redirection when ownership changes. Data movement requires coordination so concurrent requests do not disappear between old and new owners. Multi-key operations have placement constraints that should be checked in the selected API.

Adding shards helps when work can be divided. One popular leaderboard, global counter, or reservation key remains a hotspot if all updates require one authoritative sequence. Splitting a counter into partial counters improves write distribution but changes how an exact total is read. A globally bounded stock count needs more than summing eventually updated fragments if overselling is prohibited.

An illustrative workload sends 100,000 commands/s across ten shards, but 60,000 commands/s target one key. The other 40,000 can distribute to roughly 4,000 per shard. The hot owner still receives its 60,000 plus ordinary work. Average cluster utilization hides the saturated authoritative path.

Possible redesigns include partitioning by independent item, separating approximate popularity from exact inventory, batching safe increments, or allocating escrow rights. Every option changes either the work shape or the semantics. “Add nodes” is not a complete response to an invariant that demands serialization on one object.

A worked deployment comparison

Consider two hypothetical uses with identical 5 ms application latency targets. The first stores product-card summaries that can be rebuilt from Postgres. The second stores the only record of purchased virtual items.

For summaries, a cache miss can return a bounded stale result or fetch from the origin. Eviction is acceptable, restart can discard cached state, and asynchronous replication may be enough. The important risks are origin overload, invalidation races, large responses, and warm-up time. Optimizing hit rate and fallback admission can improve user experience more than strengthening persistence.

For unique items, the invariant demands that successful purchases survive the stated failure model and that concurrent buyers cannot both own the same item. The architecture must specify atomic update boundaries, durable acknowledgment, replica behavior, and retry identity. If the selected Redis configuration cannot provide the needed failure contract, another authoritative engine should hold ownership while Redis serves derived views or temporary acceleration.

This is not a universal prohibition on authoritative memory-oriented databases. Memory residency and durable correctness are compatible. The point is to evaluate the actual commit, replication, and eviction rules rather than infer guarantees from performance or product category.

Diagnostic trace: finding the expensive command

A hypothetical service reports 2 ms median responses and 120 ms p99 responses. The team first suspects the network. Trace alignment instead shows that many unrelated requests become slow during the same brief interval. Their transport times are ordinary, but their server queue waits rise together. One command builds a large sorted result immediately before the interval.

The investigation separates service time from response transmission. Suppose the command spends 40 ms walking and assembling its result, then enqueues 8 MB of response bytes. At an effective 100 MB/s connection rate, draining that payload takes another 80 ms. These durations do not necessarily block the same execution resource: the 40 ms computation may delay command execution, while the 80 ms drain occupies connection buffers and network capacity. Treating the entire 120 ms as CPU time would suggest the wrong remedy.

During the 40 ms compute interval, assume ordinary requests arrive at 20,000/s. About 20,000 × 0.040 = 800 requests arrive. Even if their average service time is only 10 microseconds, clearing those 800 requires another 8 ms of serialized compute before considering new arrivals. One expensive command therefore affects many short commands. The effect is workload-dependent, and a real event loop’s scheduling and I/O behavior must be measured.

A narrower range query, a paginated interface with explicit snapshot semantics, or an asynchronously maintained summary may reduce the expensive operation. Pagination should not silently promise a stable whole-result view if concurrent updates can move entries between pages. A continuation token needs a contract: a snapshot version, stable ordered key, or an explicitly best-effort feed. Performance work can introduce correctness changes if the original interface is misunderstood.

The scenario also explains why average command complexity is insufficient. If 99.99% of operations are tiny and the remaining operations monopolize the same execution resource, tail latency depends on those rare operations and their arrival timing. Command-level tracing, response-byte accounting, and queue metrics establish which resource is actually shared.

Diagnostic trace: rebuilding a cold cache safely

Suppose a cache normally serves 99% of 100,000 reads/s. The origin sees about 1,000 misses/s. A restart removes most cached entries, and the origin suddenly receives nearly 100,000 lookups/s. If its safe capacity is 5,000/s, the nominally redundant cache restart creates a twentyfold overload relative to that capacity.

A safe rebuild limits origin work independently of client demand. Requests for the same absent key can share one in-flight fill. Frequently accessed keys can be warmed first, while low-priority requests receive bounded stale responses or explicit temporary failure if the product permits it. Randomized expiry reduces synchronized expiration, but does not replace a global origin concurrency limit.

The cache can admit new entries selectively. A large one-time reporting result may evict useful small interactive entries while offering little reuse. Caching every response is not inherently efficient. Admission based on frequency, size, and reconstruction cost can preserve the working set, subject to the operational complexity justified by the workload.

A cache is therefore a load-shaping subsystem as well as a lookup accelerator. Its healthy-state hit rate, cold-start demand, invalidation protocol, and origin protection together determine whether it improves availability. This is the same pattern encountered in replica catch-up and compute-storage disaggregation: transitional behavior can require more capacity than steady operation.

Synthesis

Memory-oriented designs are powerful when locality, native operations, and short execution paths match the workload. Their limits become visible in command duration, memory representation, background persistence, response delivery, failover semantics, and indivisible hot keys.

The next lesson extends the same discipline to distributed transactional databases. Instead of asking whether a system is “fast” or “distributed,” it follows timestamps, replication, placement, and commit decisions to determine which costs are necessary for the selected correctness contract.

Sources and further reading

• Redis latency documentation
• Redis persistence documentation
• Redis Cluster specification
• Redis key eviction

Course index · Lesson plan