Store Crates at a Glance
One row per store, with the contract columns that differ between them. The prose behind every column is in Remote Stores; each store also has its own chapter here — etcd, Consul, NATS, Redis, Vault, S3, Firestore, git — and its README has the whole story.
The async family — a watch is a future
Cancelled by dropping the future, on any executor. Stop latency is immediate for the streaming protocols; S3 is async but has no change stream, so it polls.
| Crate | Store | Watches by | Worst case for noticing a stop | Startup delivery | Deleted key | Transport failure |
|---|---|---|---|---|---|---|
dynamic-config-etcd | etcd v3 | a watch stream | immediate — the future is cancelled | not delivered; fetch first | not a change | an expired token re-authenticates and resumes from the last delivered revision; other stream errors end the watch |
dynamic-config-nats | NATS JetStream KV | a KV change stream | immediate — the future is cancelled | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-s3 | S3, and anything speaking it | polling the ETag | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
The blocking family — a watch is a thread
A thread cannot be dropped from outside, so it takes a Watching token and
checks it between requests; dropping the matching RemoteWatch stops it.
| Crate | Store | Watches by | Worst case for noticing a stop | Startup delivery | Deleted key | Transport failure |
|---|---|---|---|---|---|---|
dynamic-config-consul | Consul KV | a blocking query | the blocking query's wait, one minute by default | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-redis | Redis | keyspace notifications | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | fetch failures retry; a dead subscription ends the watch |
dynamic-config-vault | Vault KV v2 | polling the version | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-firestore | Firestore | polling updateTime | a quarter second, whatever the poll interval is | not delivered; fetch first | not a change | backs off and retries |
dynamic-config-git | a git repository | polling the ref advertisement | a quarter second, whatever the poll interval is | not delivered; fetch first | a deleted file fails the read | backs off and retries; a refused credential ends the watch |
Watching a set is the column these tables leave out, because it splits the
family a different way. Four can do it — Consul and etcd by prefix, Redis by a
named list, git by anything — and four refuse: NATS, Vault, S3 and Firestore,
along with a Redis prefix. The rule is the same everywhere. A watch on a set
needs the store to say the set changed and the set to be re-readable as of
one instant; where the second half fails, waking on one key and re-reading the
rest collects one key's new value beside another's old one — a document that
never existed, installed unprompted. The four that qualify each have an
answer: a recursive blocking query's reply is the subtree at one index, a
range read at the event's own revision is one revision, an MGET is one
command, and a git fetch resolves one commit whose tree holds every path.
The cost runs the other way instead, and is documented as spurious, never torn: a delivery may carry a state newer than the write that woke it, and a commit touching nothing the source reads still moves the ref.
The shared contract
The startup-delivery and deleted-key columns are identical on purpose — they are decisions rather than accidents, and they hold across all eight store crates. Transport failures retry too, except where the table names an error that ends the watch, so a supervisor can restart it:
- The current value is not delivered at startup. A watch reports changes; announcing the value the caller already has would make every restart look like an edit. Fetch first if the starting value matters — it usually does.
- A deleted key is not a change. No configuration is not a configuration, and neither replaying the last one nor pushing emptiness is better than leaving the running snapshot alone.
- A transport failure retries rather than ending the watch — with the
table's named exceptions: an etcd stream error no token refresh can cure,
and a Redis subscription that died, both of which end the watch with an
error. An error from your callback always ends it, so a caller that wants
to survive a bad document should log it and return
Ok.
Credential handling is also uniform: logging in is lazy, expiry is handled both before and after a request (with exactly one retry), and a credential read from a file is re-read at every login. See Credentials, and keeping them working.