Python authoring & Airflow compatibility
Leoflow does not invent a new Python API. You author DAGs in standard Apache
Airflow Task SDK code — the same DAG, @task, operators, sensors, and hooks
you already know — and Leoflow adds only a thin runtime and a packaging file on
top. That is the whole compatibility story, and it is why Leoflow’s own Python
surface is deliberately tiny.
The Python you write is Airflow’s, not Leoflow’s. Operators, sensors, hooks,
the DAG/@task API — all of that is the Apache Airflow Task SDK, documented on
airflow.apache.org. Leoflow contributes a Go
control plane (no Python), a leoflow.yaml packaging file, and a ~2-function
runtime shim. Nothing more.
What you import is Airflow
A dag.py is real Apache Airflow SDK 3.2.x code:
from airflow.sdk import DAG, task
Everything reachable from there is the upstream Airflow authoring surface, and Leoflow re-implements none of it:
- Operators & sensors —
SnowflakeOperator,S3KeySensor, the ~1,500 provider classes — are the Airflow provider ecosystem. Leoflow executes the real class inside your task pod rather than shipping its own copy. See Operators & sensors for the execution model and the current support surface, and ADR 0040 for the rollout. - Hooks & Connections — provider hooks read the standard
AIRFLOW_CONN_*wire format; the control plane delivers the secret to the pod (Variables & Connections, ADR 0021). - The run context —
{{ ds }},params, the data interval, templating — behaves as Airflow authors expect.
For the “big” reference a developer reaches for — the operator catalog, hook arguments, templating fields — the source of truth is Airflow’s own docs, not this site. Leoflow does not duplicate them.
What Leoflow adds
The Leoflow-specific surface is small and worth learning once:
| Piece | What it is | Where it’s documented |
|---|---|---|
leoflow.yaml | Packaging & deploy config that pairs with dag.py. Not an Airflow file. | DAG authoring |
| Compile-to-artifact | dag.py + leoflow.yaml → an immutable dag.json + image (ADR 0003) — parsed once, at compile time. | DAG authoring |
leoflow_runtime | The in-pod shim that runs your task callable and bridges its return value to XCom — public API is just run and xcom_pull. | Python runtime API |
| Go control plane | Speaks the Airflow-compatible /api/v2/ and orchestrates pods. Never imports Airflow. | Architecture |
Why the Python surface is small — by design
If you come from Airflow, its Python docs are enormous: operators, hooks, executors, the scheduler, the metadatabase. Leoflow’s equivalent looks like almost nothing — and that is the point, not a gap.
Leoflow’s bet is to reuse Airflow’s authoring surface rather than reimplement it. Building a hand-written runtime shim was prototyped and deliberately shelved (ADR 0036): growing it provider-by-provider is unbounded maintenance, and the real Airflow Task SDK already makes provider hooks work today. So Leoflow keeps the real SDK at the two edges (a compile-time structural shim in ADR 0024, real execution in the pod) and adds only the glue.
The consequence: there is no large Leoflow Python API to learn. You learn
Airflow (which you may already know) plus leoflow.yaml and two runtime helpers.
Where to go next
- DAG authoring — the two-file model and the compile lifecycle. Start here.
- Operators & sensors — how any provider operator runs on Leoflow.
- Python runtime API — the
run/xcom_pullreference (the Leoflow delta). - Apache Airflow docs — the operator, hook, and Task SDK reference Leoflow relies on.