Agent credential transport

Every task pod runs a Leoflow agent (ADR 0004) that talks to the control plane over gRPC to fetch the task spec, resolve the task’s Variables and Connections, push XCom, and report state. To do that the agent needs a bearer credential the control plane trusts. How that credential reaches the pod is the agent token transport, selected by auth.agent_token_transport — operator-scoped, never DAG-author-settable.

There are two transports: envvar (the default; today’s behavior, byte-for-byte) and exchange (the projected-ServiceAccount-token path). The exchange transport is a prerequisite for warm worker pools.

envvar — the plaintext bearer (default)

With auth.agent_token_transport=envvar, the control plane mints a Leoflow JWT at dispatch and sets it as a plaintext LEOFLOW_AGENT_TOKEN environment variable on the task pod’s spec. This is the historical behavior and remains the default so that nothing regresses.

The in-pod agent reads the variable; an agent-only environment strip removes it (and every other LEOFLOW_-prefixed variable outside a tiny allowlist) before user code runs, so task code cannot read the token from its own process environment.

The residual exposure this transport cannot close: the token sits in plaintext on the Pod object in etcd. Any principal with get pods in the task namespace can read it (kubectl get pod -o yaml, the Kubernetes API, any controller or audit sink that logs pod specs). The in-process strip closes the task’s own read path; it does nothing for the cluster read path. That, plus the fact that a single per-pod credential cannot carry a per-attempt identity, is why warm pools cannot use this transport.

exchange — projected SA token + TokenReview

With auth.agent_token_transport=exchange, no bearer credential sits on the pod object at all. Instead the pod mounts a projected ServiceAccount token (audience leoflow-control-plane, pod-bound, short-lived, auto-rotated by the kubelet). The agent presents that token once, and the control plane exchanges it for a task-scoped Leoflow JWT:

flowchart LR
  subgraph POD["Task pod"]
    SA["Projected SA token<br/>(audience leoflow-control-plane,<br/>pod-bound, short-lived)"]
    AG["Agent"]
    SA -->|mounted| AG
  end
  AG -->|"1 · Register<br/>(presents SA token)"| CP["Control plane"]
  CP -->|"2 · TokenReview<br/>(once per pod)"| API["Kubernetes<br/>API server"]
  API -->|"authenticated:<br/>pod-name + pod-uid"| CP
  CP -->|"3 · resolve pod → task instance<br/>via pod annotation"| CP
  CP -->|"4 · minted task-scoped<br/>Leoflow JWT"| AG
  AG -->|"5 · steady-state RPCs<br/>(secrets, XCom, heartbeat)<br/>— apiserver-free"| CP

1. Register. At Register the agent presents the projected SA token.
2. TokenReview — once per pod. The control plane calls the Kubernetes TokenReview API to validate the token. This call lands inside the window the API server was already required to create the pod, so the net added API-server coupling is ≈ 0. (Doing a TokenReview on every secret fetch was explicitly rejected — it would load the API-server budget on the secret hot path, and an SA token identifies the ServiceAccount, not the attempt.)
3. Resolve pod → task instance. From the authenticated pod identity the control plane resolves the exact task instance via the pod’s identity annotation (pod labels are sanitized and lossy, so the annotation is the single-sourced contract).
4. Mint a task-scoped JWT. The control plane returns a task-scoped Leoflow JWT — the identity that secret scoping filters on and that liveness enforcement checks.
5. Apiserver-free steady state. Every steady-state RPC (secrets, XCom, heartbeat) authenticates with that JWT. The projected SA token was a bootstrap credential exchanged exactly once, not a per-fetch capability — so the secret hot path never touches the API server.

The invariant this satisfies is unconditional: no bearer credential is ever a plaintext field on the Pod object. (For a cluster that cannot project a ServiceAccount token, a SecretKeyRef fallback sources the credential from a Kubernetes Secret rather than a plaintext value — still off the plaintext pod spec.)

The two-token model

Under a warm pool the exchange cleanly separates a pod-scoped credential from an attempt-scoped one — the property that makes reusing a pod across attempts safe:

flowchart TB
  subgraph BOOT["Bootstrap token · worker-scoped"]
    direction TB
    B1["Minted by: the projected-SA exchange<br/>(one TokenReview at Register)"]
    B2["Held by: the warm pod, for its whole life<br/>(short TTL, heartbeat-renewed)"]
    B3["Authorizes: Register + AwaitAssignment ONLY"]
    B4["Fetches NO secrets"]
  end
  subgraph TASK["Task-scoped token · per-attempt"]
    direction TB
    T1["Minted by: the control plane, per attempt"]
    T2["Held by: one attempt, for its lifetime<br/>(short TTL, renewed on heartbeat)"]
    T3["Authorizes: THAT attempt's declared<br/>secrets + XCom + state report"]
    T4["Liveness-gated: dies the instant its<br/>task instance is no longer live"]
  end
  BOOT -.->|"the pod bootstraps,<br/>then each attempt gets<br/>its own task token in-band"| TASK

The task-scoped token is what “dies with the attempt”: because it is minted per attempt, short-lived, and gated on task-instance liveness, a finished or superseded attempt’s token stops resolving secrets regardless of the clock. On a reused pod that is exactly what stops one attempt’s credential from bleeding into the next. This blast-radius argument holds only when secret_liveness_mode=enforce; under the observe default a superseded attempt’s token would still resolve secrets. That is why warm pools require both agent_token_transport=exchange and secret_liveness_mode=enforce, validated fail-closed at boot — see Warm worker pools → How to enable.

RBAC — the tokenreviews grant

The TokenReview call requires a cluster-scoped RBAC grant (create on authentication.k8s.io/tokenreviews) for the control plane’s ServiceAccount. Token review is inherently cluster-scoped — it is not namespaced — so this is a ClusterRole + ClusterRoleBinding, not a namespaced Role.

The Helm chart renders that ClusterRole and binding only when the exchange transport is selected. On the default envvar transport nothing cluster-scoped is created, so a default install grants the control plane no cluster-wide permission it does not need. Selecting exchange is what opts the deployment into the tokenreviews grant.

Relationship to secret scoping

The transport is one of four related knobs from ADR 0055. They narrow different dimensions of the same credential and are configured independently:

Warm pools require agent_token_transport=exchange and secret_liveness_mode=enforce. secret_scoping and max_attempt_credential_lifetime are independent hardening choices with their own rollout arcs. All are operator-scoped and documented in the Configuration reference.

See also

• Warm worker pools — the feature that requires the exchange transport.
• Variables & Connections — what the task-scoped token resolves.
• ADR 0055 — Secret scoping and token liveness.
• ADR 0004 — The thin agent.
• Configuration reference.