Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The Three Deliveries: File, Env, Secret

One engine, three ways a document reaches a workload — because real software disagrees about how it wants to be configured. Grafana re-reads files; Airflow reads environment variables at boot and nothing else; a Strimzi-shaped operator watches Kubernetes Secrets. Picking the delivery is a one-line decision here; what follows is the map.

File (webhook + agent)Env (env-inject)Secret (operator target)
The consumerreads/watches a filereads environ at startreads/watches a k8s Secret, or envFrom
Freshnesslive — atomic rename, watcher cadencefrozen at container start (Kubernetes' rule); env-restart opts into a kubelet container-restart on change — seconds, no pod recreationlive object; watchers react, envFrom at next start
Touches etcd?never — tmpfs emptyDirnever — same tmpfs file, sourcedyes — a Secret lives in etcd, stated out loud
Restart to update?noyes (next pod start)no for Secret-watchers; yes for envFrom
Set up bypod annotationspod annotations (+ explicit command)DynamicConfigRender CR
Real exampleGrafanaAirflowKafka client

The decision procedure, in order: can the consumer read a file? File delivery — it is the only fully live one and the only one that never touches etcd. Does something else own the consumer (an operator that only reads Secrets)? The Secret target. Env-only software? env-inject, with the start-freeze stated rather than papered over.

Against the Vault Agent Injector

The closest relative of the webhook+agent half — same architecture (mutating webhook, injected init/sidecar, shared memory volume, pod service-account auth, file permissions and run-as knobs), different center of gravity:

Vault Agent Injectordynamic-config-k8s
BackendsVaultnine stores — Vault among them, plus Consul, etcd, git, Redis, NATS, S3, Firestore, config-server
Pod authKubernetes auth (SA token)the same, wherever the store speaks it (Vault, Consul); IRSA/Workload Identity for S3/Firestore; secret-based stays first-class where a store never will
What is deliveredrendered secrets, Consul-Template languagethe resolved configuration document — precedence, validation, provenance — in json/toml/yaml/ini/properties or a minijinja template
File perms / ownershipannotationsannotations (file-mode, agent-run-as-user/group), root refused at admission
Env variablesyou rewrite the command by hand to source the fileenv-inject writes the wrap for you, refuses the impossible cases by name
k8s Secret objectsnothe operator's secret: target, when the consumer requires one
Lease renewalvault-agent renews leasesthe agent re-fetches on its interval; Vault reads are versioned-metadata-gated
Scopesecrets deliveryconfiguration delivery that treats secrets as first-class fields

The injector's template idiom, translated

The Vault injector spells "render me a connection string" as a per-secret annotation pair; here the same result is one source and one template, because the whole pod has one resolved document:

# Vault Agent Injector:
#   vault.hashicorp.com/agent-inject-secret-db-creds: "secret/data/db-app"
#   vault.hashicorp.com/agent-inject-template-db-creds: |
#     {{- with secret "secret/data/db-app" -}}
#     postgres://{{ .Data.data.username }}:{{ .Data.data.password }}@postgres:5432/appdb
#     {{- end }}

# dynamic-config-k8s, the same string from the same KV secret:
dynamic-config.rs/source: "vault"
dynamic-config.rs/endpoint: "https://vault.vault.svc:8200"
dynamic-config.rs/key: "secret/db-app"
dynamic-config.rs/auth: "kubernetes"
dynamic-config.rs/auth-role: "db-app"
dynamic-config.rs/path: "/config/db.env"
dynamic-config.rs/template: |
  DATABASE_URL=postgres://{{ username }}:{{ password }}@postgres:5432/appdb

The template owns the bytes (minijinja, strict-undefined: a typo is an error, not an empty string), so any shape works — a URL, an .env, a whole config file. What does NOT translate is the database/creds/… path in the injector's example: that is Vault's dynamic secrets engine, credentials minted per-request with leases — the boundary the next paragraph prices. This agent reads KV; for minted-with-TTL credentials, run the Vault Agent beside it.

One more idiom, matched: the injector's several -secret-<name> pairs per pod are this webhook's named renderssource.db, key.db, path.db beside the default, one agent and one file per name, all in one shared directory. When the pod wants them MERGED into a single document instead, the config server composes sections and the pod reads one endpoint. What has no counterpart is agent-inject-command (a post-render hook): env-restart covers the restart case, and anything richer belongs to the app.

Honest edge the other way: for dynamic Vault secrets with leases (database credentials minted per-pod, TTL renewal mid-life), the Vault Agent is the purpose-built tool and this is not — this agent re-fetches documents; it does not manage leases.

Against External Secrets Operator

The closest relative of the operator half — same split between a namespaced store and a platform-owned cluster store, different product:

External Secrets Operatordynamic-config-k8s
Store definitionSecretStore / ClusterSecretStoreDynamicConfigClass / ClusterDynamicConfigClass — the same two scopes, allowlist included
Outputa Kubernetes Secret, alwaysa ConfigMap, a Secret, or a file no etcd ever sees
The etcd tradeevery delivered secret lives in etcdonly the Secret target does, and choosing it is explicit — the file path exists precisely to avoid it
Data modelkey-by-key secret mappingwhole configuration documents: precedence, validation, formats, provenance
Env deliveryenvFrom the Secretthe same via envEntries — or env-inject, which needs no Secret at all
Backendsvery many secret managersnine configuration stores
TemplatingSecret templatesminijinja over the resolved document

Honest edge the other way: as a secret-synchronisation fleet tool across dozens of managers (AWS/GCP/Azure SM, Doppler, 1Password…), ESO has breadth this project does not chase — the store list here grows by demand, not by roadmap.

Use cases, mapped

  • Airflow / env-only softwareenv-inject over a rendered dotenv (example); add env-restart: "true" and a changed document restarts just that container in seconds — otherwise changes wait for the next pod start, and that limit is stated instead of hidden.
  • Grafana / anything that re-reads files → the sidecar; live updates, zero etcd, tmpfs only (example).
  • Strimzi-shaped operators / JVM client.properties → the Secret target, file or envEntries shape (example).
  • Multi-tenant platformsClusterDynamicConfigClass with a namespaces allowlist; tenants never see a credential (example).
  • A chart's existingSecret / an operator's secretName: → the Secret target with shape: entries — leaf keys verbatim, so the names some other chart already chose are met exactly (example).
  • All of it at once → the four-component shop stack: three secrets injected three ways (a chart's existingSecret, a secretKeyRef env, a mounted-and-live file), the API on env-inject + env-restart, the worker on live files — credentials existing only in the platform namespace.
  • Vault dynamic database credentials with TTLs → the Vault Agent Injector, genuinely. Use both: it owns the lease, this owns the configuration.