Store Crates at a Glance
One row per store, with the contract columns that differ between them. The prose behind every column is in Remote Stores; each store also has its own chapter here — etcd, Consul, NATS, Redis, Vault, S3, Firestore, git — and its README has the whole story.
The async family — a watch is a future
Cancelled by dropping the future, on any executor. Stop latency is immediate for the streaming protocols; S3 is async but has no change stream, so it polls.
| Crate | Store | Watches by | Worst case for noticing a stop | Startup delivery | Deleted key | Transport failure |
|---|---|---|---|---|---|---|
dynamic-config-etcd | etcd v3 | a watch stream | immediate — the future is cancelled | not delivered; fetch first | not a change | an expired token re-authenticates and resumes from the last delivered revision; other stream errors end the watch |
dynamic-config-nats | NATS JetStream KV | a KV change stream | immediate — the future is cancelled | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-s3 | S3, and anything speaking it | polling the ETag | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
The blocking family — a watch is a thread
A thread cannot be dropped from outside, so it takes a Watching token and
checks it between requests; dropping the matching RemoteWatch stops it.
| Crate | Store | Watches by | Worst case for noticing a stop | Startup delivery | Deleted key | Transport failure |
|---|---|---|---|---|---|---|
dynamic-config-consul | Consul KV | a blocking query | the blocking query's wait, one minute by default | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-redis | Redis | keyspace notifications | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | fetch failures retry; a dead subscription ends the watch |
dynamic-config-vault | Vault KV v2 | polling the version | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-firestore | Firestore | polling updateTime | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-git | a git repository | polling the ref advertisement | a quarter second, whatever the poll interval is | not delivered; fetch first | a deleted file fails the read | backs off and retries; a refused credential ends the watch |
Watching a set is the column these tables leave out, because it splits the
family a different way. Four can do it — Consul and etcd by prefix, Redis by a
named list, git by anything — and four refuse: NATS, Vault, S3 and Firestore,
along with a Redis prefix. The rule is the same everywhere. A watch on a set
needs the store to say the set changed and the set to be re-readable as of
one instant; where the second half fails, waking on one key and re-reading the
rest collects one key's new value beside another's old one — a document that
never existed, installed unprompted. The four that qualify each have an
answer: a recursive blocking query's reply is the subtree at one index, a
range read at the event's own revision is one revision, an MGET is one
command, and a git fetch resolves one commit whose tree holds every path.
The cost runs the other way instead, and is documented as spurious, never torn: a delivery may carry a state newer than the write that woke it, and a commit touching nothing the source reads still moves the ref.
The shared contract
The startup-delivery and deleted-key columns are identical on purpose — they are decisions rather than accidents, and they hold across all eight store crates. Transport failures retry too, except where the table names an error that ends the watch, deliberately, so a supervisor can restart it:
- The current value is not delivered at startup. A watch reports changes; announcing the value the caller already has would make every restart look like an edit. Fetch first if the starting value matters — it usually does.
- A deleted key is not a change. No configuration is not a configuration, and neither replaying the last one nor pushing emptiness is better than leaving the running snapshot alone.
- A transport failure retries rather than ending the watch — with the
table's named exceptions: an etcd stream error no token refresh can cure,
and a Redis subscription that died, both of which end the watch with an
error. An error from your callback always ends it, so a caller that wants
to survive a bad document should log it and return
Ok.
Credential handling is also uniform: logging in is lazy, expiry is handled both before and after a request (with exactly one retry), and a credential read from a file is re-read at every login. See Credentials, and keeping them working.