Course contents
On this page
Do not lose the update between loading and listening
Join an initial snapshot to a change stream at an explicit position.
A shop dashboard loads stock 10, then opens a subscription for updates. Between those two actions, a purchase changes stock to 9. If the subscription begins after that change, the dashboard can remain at 10 indefinitely.
A persistent connection does not repair the gap. We need a defined relationship between the initial state and the changes that follow it.
Choose a boundary both sides understand
Use a teaching stream that gives committed transactions increasing positions. The server can produce a consistent snapshot corresponding exactly to position 100 and retain every later change until the client can consume it. These are required capabilities of this model, not properties of arbitrary event feeds.
- Snapshot at position 100
The dashboard installs stock 10 from that snapshot.
- Change at position 101
A purchase sets stock to 9. The retained stream includes it.
- Resume after position 100
The dashboard applies change 101, even if it subscribed after the purchase.
The position is sometimes called a watermark or cursor. It must describe an actual visibility boundary, not merely the wall-clock time when an HTTP request began. A snapshot containing only some transactions below the claimed boundary would break the reasoning.
When applying a change, the consumer records its progress consistently with the resulting state. Reconnecting after 100 may replay 101. A change saying “subtract one” would double-apply unless the consumer recognizes that it already processed that position. A complete replacement value can simplify replay, but ordering still matters: an older value must not overwrite a newer one.
Let retention define recovery
The server cannot necessarily retain every change forever. Suppose a disconnected client last processed position 100, but the oldest retained change is now 150. Resuming at 150 would silently omit positions 101–149.
The protocol should instead signal that the cursor is too old and require a fresh aligned snapshot. Slow consumers need bounded buffers and a documented recovery path. Simply allocating more memory moves the failure later.
A filtered view also needs removals. If the dashboard shows only products with stock above zero, a change from 1 to 0 must remove the product from that view, not disappear because its new value fails the filter.
Predict a reconnect
The client applies change 101 but crashes before recording progress beyond 100. What must it expect when reconnecting?
Account for replay
It may receive 101 again. It must either restore state and progress together or apply replay safely using durable change identities and ordering rules. Assuming “the server sent it once” does not establish “the client applied it once.”
Source: PostgreSQL 18 logical decoding concepts, including exported snapshots and possible replay after crashes. Actual stream protocols expose their own position and retention contracts.
Extended reference
The extended chapter preserves the original technical coverage, additional examples, and citations. The explanation above supplies the concepts needed for this page; the reference develops the specialist cases.
Find a section from the original chapter
- Learning outcomes
- The missing update between snapshot and subscription
- Events and state are different interfaces
- A watermark aligns snapshot and changes
- Ordering and duplicate handling
- Fan-out arithmetic
- Slow consumers and bounded state
- Permissions during long-lived subscriptions
- Case comparison
- A complete snapshot-and-stream trace
- Projection changes include removals
- Join maintenance can amplify one small write
- Client application is another execution stage
- Backlog policies follow semantics
- Choosing an abstraction through a product scenario
- A practical latency budget for delivery
- Reconnect storms are a distinct workload
- Design conclusion
- Sources and further reading