DAG authoring
Author a DAG: leoflow.yaml plus dag.py compiled to one immutable artifact.
Version v0.4.0 of the documentation is no longer actively maintained. The site that you are currently viewing is an archived snapshot. For up-to-date documentation, see the latest version.
Author DAGs
Author DAGs on the Airflow SDK — operators, dbt, variables & connections, alerting, map-reduce, and worked examples.
Everything about writing DAGs for Leoflow. A DAG is a dag.py on the Airflow Task
SDK plus a leoflow.yaml for packaging and bindings, compiled to one immutable
artifact.
- DAG authoring — the project layout, the two files, and the compile model.
- Operators & sensors — use Airflow operators and sensors from your tasks.
- dbt projects as DAGs — render a dbt project into model-level tasks.
- Variables & Connections — expose Variables and Connections to your task pods.
- On-failure alerting — notify on run failure from
leoflow.yaml, no extra task. - Map-reduce for ML — fan-out + reduce as a Python list comprehension.
- Examples — runnable example DAGs.
- ETL case study — a worked 1 GB ETL on the staging volume.
- The Lite web editor — edit and run DAGs from the browser.
Airflow operators & sensors
Use Airflow operators and sensors from your DAGs on Leoflow.
dbt projects as DAGs
Render a dbt project into a Leoflow DAG with native model-level tasks.
Variables & Connections
Expose Variables and Connections to your task pods.
On-failure alerting
Notify on run failure from leoflow.yaml — Slack or a generic webhook, no extra task and no Python.
Map-reduce for ML
Fan-out plus reduce as a Python list comprehension — native map-reduce for ML/AI.
Examples
Runnable example DAGs covering the common authoring patterns.
Case study: 1 GB ETL on staging
A worked 1 GB ETL that shares data between tasks through the per-run staging volume.
The Lite web editor
Edit and run DAGs from the browser in Leoflow Lite.