Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Store Crates at a Glance

One row per store, with the contract columns that differ between them. The prose behind every column is in Remote Stores; each store also has its own chapter here — etcd, Consul, NATS, Redis, Vault, S3, Firestore, git — and its README has the whole story.

The async family — a watch is a future

Cancelled by dropping the future, on any executor. Stop latency is immediate for the streaming protocols; S3 is async but has no change stream, so it polls.

CrateStoreWatches byWorst case for noticing a stopStartup deliveryDeleted keyTransport failure
dynamic-config-etcdetcd v3a watch streamimmediate — the future is cancellednot delivered; fetch firstnot a changean expired token re-authenticates and resumes from the last delivered revision; other stream errors end the watch
dynamic-config-natsNATS JetStream KVa KV change streamimmediate — the future is cancellednot delivered; fetch firstnot a changebacks off and retries
dynamic-config-s3S3, and anything speaking itpolling the ETaga quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries

The blocking family — a watch is a thread

A thread cannot be dropped from outside, so it takes a Watching token and checks it between requests; dropping the matching RemoteWatch stops it.

CrateStoreWatches byWorst case for noticing a stopStartup deliveryDeleted keyTransport failure
dynamic-config-consulConsul KVa blocking querythe blocking query's wait, one minute by defaultnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-redisRediskeyspace notificationsa quarter second, whatever the poll interval isnot delivered; fetch firstnot a changefetch failures retry; a dead subscription ends the watch
dynamic-config-vaultVault KV v2polling the versiona quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-firestoreFirestorepolling updateTimea quarter second, whatever the poll interval isnot delivered; fetch firstnot a changebacks off and retries
dynamic-config-gita git repositorypolling the ref advertisementa quarter second, whatever the poll interval isnot delivered; fetch firsta deleted file fails the readbacks off and retries; a refused credential ends the watch

Watching a set is the column these tables leave out, because it splits the family a different way. Four can do it — Consul and etcd by prefix, Redis by a named list, git by anything — and four refuse: NATS, Vault, S3 and Firestore, along with a Redis prefix. The rule is the same everywhere. A watch on a set needs the store to say the set changed and the set to be re-readable as of one instant; where the second half fails, waking on one key and re-reading the rest collects one key's new value beside another's old one — a document that never existed, installed unprompted. The four that qualify each have an answer: a recursive blocking query's reply is the subtree at one index, a range read at the event's own revision is one revision, an MGET is one command, and a git fetch resolves one commit whose tree holds every path.

The cost runs the other way instead, and is documented as spurious, never torn: a delivery may carry a state newer than the write that woke it, and a commit touching nothing the source reads still moves the ref.

The shared contract

The startup-delivery and deleted-key columns are identical on purpose — they are decisions rather than accidents, and they hold across all eight store crates. Transport failures retry too, except where the table names an error that ends the watch, deliberately, so a supervisor can restart it:

  • The current value is not delivered at startup. A watch reports changes; announcing the value the caller already has would make every restart look like an edit. Fetch first if the starting value matters — it usually does.
  • A deleted key is not a change. No configuration is not a configuration, and neither replaying the last one nor pushing emptiness is better than leaving the running snapshot alone.
  • A transport failure retries rather than ending the watch — with the table's named exceptions: an etcd stream error no token refresh can cure, and a Redis subscription that died, both of which end the watch with an error. An error from your callback always ends it, so a caller that wants to survive a bad document should log it and return Ok.

Credential handling is also uniform: logging in is lazy, expiry is handled both before and after a request (with exactly one retry), and a credential read from a file is re-read at every login. See Credentials, and keeping them working.