Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Store Crates at a Glance

One row per store, with the contract columns that differ between them. The prose behind every column is in Remote Stores; each store also has its own chapter here — etcd, Consul, NATS, Redis, Vault, S3, Firestore, git — and its README has the whole story.

The async family — a watch is a future

Cancelled by dropping the future, on any executor. Stop latency is immediate for the streaming protocols; S3 is async but has no change stream, so it polls.

CrateStoreWatches byWorst case for noticing a stopStartup deliveryDeleted keyTransport failure
dynamic-config-etcdetcd v3a watch streamimmediate — the future is cancellednot delivered; fetch firstnot a changean expired token re-authenticates and resumes from the last delivered revision; other stream errors end the watch
dynamic-config-natsNATS JetStream KVa KV change streamimmediate — the future is cancellednot delivered; fetch firstnot a changebacks off and retries
dynamic-config-s3S3, and anything speaking itpolling the ETaga quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries

The blocking family — a watch is a thread

A thread cannot be dropped from outside, so it takes a Watching token and checks it between requests; dropping the matching RemoteWatch stops it.

CrateStoreWatches byWorst case for noticing a stopStartup deliveryDeleted keyTransport failure
dynamic-config-consulConsul KVa blocking querythe blocking query's wait, one minute by defaultnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-redisRediskeyspace notificationsa quarter second, whatever the poll interval isnot delivered; fetch firstnot a changefetch failures retry; a dead subscription ends the watch
dynamic-config-vaultVault KV v2polling the versiona quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-firestoreFirestorepolling updateTimea quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-gita git repositorypolling the ref advertisementa quarter second, whatever the poll interval isnot delivered; fetch firsta deleted file fails the readbacks off and retries; a refused credential ends the watch

Watching a set is the column these tables leave out, because it splits the family a different way. Four can do it — Consul and etcd by prefix, Redis by a named list, git by anything — and four refuse: NATS, Vault, S3 and Firestore, along with a Redis prefix. The rule is the same everywhere. A watch on a set needs the store to say the set changed and the set to be re-readable as of one instant; where the second half fails, waking on one key and re-reading the rest collects one key's new value beside another's old one — a document that never existed, installed unprompted. The four that qualify each have an answer: a recursive blocking query's reply is the subtree at one index, a range read at the event's own revision is one revision, an MGET is one command, and a git fetch resolves one commit whose tree holds every path.

The cost runs the other way instead, and is documented as spurious, never torn: a delivery may carry a state newer than the write that woke it, and a commit touching nothing the source reads still moves the ref.

The shared contract

The startup-delivery and deleted-key columns are identical on purpose — they are decisions rather than accidents, and they hold across all eight store crates. Transport failures retry too, except where the table names an error that ends the watch, so a supervisor can restart it:

  • The current value is not delivered at startup. A watch reports changes; announcing the value the caller already has would make every restart look like an edit. Fetch first if the starting value matters — it usually does.
  • A deleted key is not a change. No configuration is not a configuration, and neither replaying the last one nor pushing emptiness is better than leaving the running snapshot alone.
  • A transport failure retries rather than ending the watch — with the table's named exceptions: an etcd stream error no token refresh can cure, and a Redis subscription that died, both of which end the watch with an error. An error from your callback always ends it, so a caller that wants to survive a bad document should log it and return Ok.

Credential handling is also uniform: logging in is lazy, expiry is handled both before and after a request (with exactly one retry), and a credential read from a file is re-read at every login. See Credentials, and keeping them working.