dagronDurable DAG workflows
in one Rust binary.
For work that is expensive to lose: training runs, fine-tuning, batch inference, agent loops. Define the graph in plain YAML; dagron validates it and runs each task the moment its dependencies succeed. An eight-hour step holds a heartbeat lease, so it is never mistaken for a dead one, and a retry resumes from its own checkpoint instead of starting over. No control plane, no cluster to operate, a database as the only state.
Apache-2.0 Single static binary SQLite or Postgres Kubernetes-native MCP server — 42 tools
| Time | Workflow | Task | Runner | Status |
|---|---|---|---|---|
| 02:00 | NIGHTLY-ETL | LOAD | DEFAULT | RUNNING |
| 02:15 | FEATURE-BUILD | FEATURES | DOCKER | CACHE HIT |
| 03:00 | LLM-FINETUNE | TRAIN | GPU-A100 | RUNNING |
| 03:40 | WAREHOUSE-ROLLUP | ROLLUP | POOL:ETL | QUEUED |
| 04:05 | CDC-ORDERS | INGEST | STREAM | RETRY 2/3 |
| 04:30 | MODEL-PUBLISH | PUBLISH | APPROVAL | AWAITING OK |
| 05:00 | SENSOR-BACKFILL | PARSE | DEFAULT | DEAD-LETTER |
Waiting / retrying Failed → dead-letter Succeeded illustrative board — not live data
The eight-hour step
is the hard part.
Anything can run a graph of short tasks. The work that costs money runs for hours on hardware you are paying for by the second, gets preempted, and must not silently begin again at zero. That is the case dagron is built around, and every mechanism below is in the open build.
Seven hours in, the node this task was running on drops. The lease stops being renewed, the scheduler reclaims the task and retries it — and because the task had reported a checkpoint, the retry starts at step 4,200 rather than at zero. The alternative is not a slower run; it is seven hours of paid GPU time spent twice.
A long task is never mistaken for a dead one.
A running task renews a heartbeat lease every 10 seconds under a triple-guarded claim, so a step may run for hours without the scheduler deciding its worker died and handing the work to someone else.
A retry picks up where the failure happened.
A task reports a checkpoint over HTTP or by file convention; on retry dagron hands it back as DAGRON_RESUME_FROM. The difference between a retry that costs a minute and one that costs a night is whether the engine remembers where you were.
Route the GPUs; cap what shares them.
Tasks carry a runner class, so the fine-tune lands on the A100 pool and the CSV parse does not. Named concurrency pools bound how many run at once, and dispatch priority decides who goes first when the pool is full.
Retries budgeted by fault class, not by count.
A node with failing ECC memory and a malformed input are not the same failure and must not get the same three attempts. Retry budgets are set per fault class, so an infrastructure fault can be retried elsewhere while a bad input stops immediately.
A conversation is a run.
The MCP server exposes 42 tools over stdio (author, run, wait, recover, inspect), so an agent drives dagron without a human dropping to curl. An agent loop is an ordinary workflow: each turn is its own child run and state is artifacts rather than process memory, so the conversation is durable, inspectable and resumable like anything else on the board.
Everything to run workflows.
Nothing you don't need.
A lean trade: one Rust binary, plain YAML, and a database — durable orchestration without the operational weight of Airflow or a Celery fleet.
A graph of tasks, and what each one waits for.
The spec is validated and cycle-checked server-side before a single task runs, so a broken graph fails at submission rather than halfway through a nightly load.
A failed task retries. A parked task holds nothing.
Per-task max_attempts with exponential backoff, timeouts and trigger rules. Approval gates and wait sensors park a run while it occupies no worker slot.
Runs stream into a live DAG view.
Inspect the graph and read per-task logs while the work is still moving — over the same engine the CLI drives.
The storage backend is a build flag.
Start on an embedded SQLite file, then move to Postgres for multi-node without touching the workflow.
Three executors, routed by runner class.
Run tasks in-process, as Docker containers, or as one-shot Kubernetes Pods — and send the expensive hardware only the steps that need it.
Reconciled from a repository, or driven by an agent.
A dedicated worker reconciles versioned workflows from a repository; the MCP server exposes submit, inspect and cancel to an AI agent over the same authenticated API the console uses.
Lineage that downstream work can act on.
produces: records which run wrote which dataset. Sense one, or fire a whole workflow when it updates.
Concurrency caps, dispatch order, and a cache key.
Cap concurrency per pool, order the ready queue by dispatch priority, and memoize a step so an unchanged key never runs again.
One binary at the edge.
The same YAML in the cluster.
The spec in the middle does not change between them. Only what it is pointed at does.
One process, one file
A static binary against a local SQLite file. State is local, so the site keeps running while disconnected and resumes from its own database after a reboot.
- State
- site.db (SQLite)
- Executor
- local subprocess
- Arch
- arm64 · amd64 · armv7
- Operates
- nothing
# workflow.yaml — unchanged in both columns name: nightly-etl tasks: - name: extract command: ["extract.sh"] max_attempts: 3 - name: transform depends_on: [extract] - name: load depends_on: [transform] produces: ["s3://dw/daily"]
Multi-node, Kubernetes
The same spec against Postgres, dispatching one-shot Pods by runner class, installed from the OCI Helm chart.
- State
- Postgres
- Executor
- Kubernetes Pods
- Install
- helm · OCI chart
- Operates
- no control plane
Six shapes it fits.
dagron scales down as well as it scales out: the workflow you run on a gateway is the workflow you run on Postgres and Kubernetes, unchanged.
AI & ML workloads
Training, fine-tuning, batch inference, agent-driven runs
- The durability mechanisms are above: heartbeat leases, checkpoint resume, GPU routing and fault-class retry budgets. See the eight-hour step.
- Gang scheduling co-schedules N ranks all-or-nothing for distributed training Enterprise; the open build runs the ranks as ordinary tasks.
- Eval gates as approvals. A run parks in awaiting_approval until a metric threshold or a person releases it, holding no worker while it waits.
- The YAML is the whole interface: the spec beside this row is the entire task.
Data engineering
Nightly loads, backfills, warehouse pipelines
- Schedules and backfills with cron, plus a dead-letter queue you can inspect and redrive.
- Fair sharing. Concurrency pools and dispatch priority stop a wide backfill starving the nightly load.
- Lineage that means something. produces: records which run wrote which dataset; downstream work can wait on it or fire from it.
- Reuse whole pipelines as a single step with template: — a DAG of DAGs, expanded at run creation.
Data science
Feature builds, evaluation, reproducible notebooks-as-jobs
- Don't recompute what hasn't changed. cache: memoizes a successful step by key; a later run with the same key reuses the output and skips execution entirely.
- A human in the loop. An approval gate parks the run, holding no worker, until someone approves or rejects it.
- Per-task environments. Each step can be its own container image with its own CPU and memory request.
Event-driven & streaming
CDC, sensor feeds, per-event pipelines
- A stream of workflows. Follow an NDJSON file or FIFO: each line submits a run; a directory of shards splits across consumers by lease.
- Exactly once. The run and the source offset commit in the same transaction, so a crash can neither duplicate a run nor lose one.
- Poison lines don't stall the feed: they dead-letter, and you redrive them once fixed.
- Managed broker connectors for Redis, SQS, Kafka and NATS Enterprise; the file and FIFO source is open.
Edge & IoT
Gateways, retail sites, vehicles, remote plant
- Nothing to operate. One static binary and a SQLite file. No control plane, no broker, no cluster.
- Small enough to co-locate. Published for arm64, amd64 and armv7, the 32-bit gateways already in the field. The engine container idles around 10 MB resident, and the whole stack on a Pi 4 peaks at ~340 MB of 3.7 GB (measured).
- One line turns the host into an edge host. profile: edge presets the constrained-host knobs; dagron config prints every one with where it came from, so a preset never hides a value from you.
- Admission gates, not crashes. DAGRON_PRESSURE_FILE lets a thermal, battery or maintenance daemon stop new work by touching a file, and DAGRON_MIN_FREE_BYTES holds a free-disk floor (64 MiB default) so a full flash refuses admission instead of corrupting a run.
- MQTT is the broker at the edge, and it is in this build. SOURCE=mqtt subscribes a topic and turns messages into runs, behind a Cargo feature rather than an edition, so a stock binary links no broker client and --features mqtt is all it takes.
- The clock is not assumed. A gateway boots without RTC or network and lies about the time, so every run carries a confidence (synced, drifted or unknown, defaulting to unknown because that is the honest answer without evidence) for your pipeline to act on.
- Signed workflow bundles. A definition pushed to a unit is ed25519-verified against DAGRON_BUNDLE_PUBKEYS before anything runs, fail-closed: unset the keys and the route answers 501 rather than accepting an unsigned spec. Verification ships in every build.
- Survives the uplink. State is local, so a site keeps running while disconnected and resumes from its own database after a reboot.
- Lint before you ship. dagron validate parses, expands and graph-checks a spec offline, with no server and no network.
Lightweight compute
Small teams, side services, anything that doesn't deserve Airflow
- Zero infra to start. dagron dev brings up the engine, the management API and Swagger on localhost.
- No idle daemon. A one-shot run drains and exits; it only stays resident when you configure it to.
- Room to grow. Move the same YAML to Postgres for multi-node: a build flag, not a rewrite.
The concepts map.
The Python does not.
The question that decides an evaluation is not "is it faster", it is "what does the port cost and what do I give up". Here is the whole answer, including the part that is a genuine loss.
What you give up: the Python DSL. A dagron workflow is data, not a program. There is no PythonOperator and no DAG file that executes at parse time. Your Python still runs, as a command, a container or a pod; what stops being Python is the graph. That is the trade the whole design rests on: a spec that cannot run arbitrary code at parse time is one that dagron validate can check offline, and that a signed bundle can be verified before it is applied.
There is no Airflow converter, and this page will not pretend otherwise — the port above is by hand. Coming from Argo Workflows there is one: the open build ships an importer that converts an entrypoint dag template into a dagron spec. The command is below. Temporal and Prefect are a different shape of tool (durable execution, and a Python-native framework); neither has an importer here.
# the importer is a workspace binary, not in the release # archives or any image — run it from a clone $ cargo run -p dagron-import \ -- argo my-argo-workflow.yaml > workflow.yaml # it refuses what it cannot map, so check what came out $ dagron validate workflow.yaml $ dagron dev workflow.yaml
One entrypoint dag template, whose tasks reference container templates, becomes a dagron spec with command, docker_image and depends_on. Steps templates, artifacts, parameters and when-expressions are reported as errors, not dropped: the importer would rather stop than hand you a spec that runs and is quietly wrong.
Free and open.
Enterprise when you scale.
The core engine, UI, and Helm chart are Apache-2.0 and free forever. The Enterprise edition adds the identity, tenancy, and operations layer teams need in production.
in this edition· not in this edition
| Capability | Open Source | Enterprise |
|---|---|---|
| DAG engine, retries, scheduling, web UI | ||
| SQLite / Postgres, Docker & Kubernetes executors | ||
| Approval gates, GitOps sync, MCP, dead-letter queue | ||
| Datasets — produces:, lineage, sensors, single-dataset triggers | ||
| Pools, dispatch priority, result memoization, wait sensors | ||
| Streaming ingestion — file/FIFO source, sharded consumers | ||
| Multi-dataset composition & external dataset events | ||
| Managed broker sources — Redis, SQS, Kafka, NATS | ||
| Gang scheduling — all-or-nothing co-scheduled ranks | ||
| Encryption at rest — envelope / BYOK-KMS + key rotation | ||
| SSO — OIDC, SAML, LDAP / Active Directory | ||
| Multi-tenant control plane & tenant router | ||
| RBAC roles + full audit trail | ||
| Auto-backfill & self-healing reruns | ||
| GitOps operator + Workflow/CronWorkflow CRDs | ||
| Air-gapped install & signed offline licensing | ||
| Usage metering & billing export | ||
| Priority support & SLA |
Running dagron across teams?
Enterprise adds single sign-on, a multi-tenant control plane with per-tenant isolation, RBAC and audit, air-gapped deployment, and a support agreement — self-hosted on your infrastructure. Metered or flat-license pricing.
The same engine.
Someone else on call.
A managed dagron: we operate the control plane, the datastore and the upgrades, and you keep writing the same YAML against the same API. The workflows you run on your laptop today are the workflows that run there — moving is a connection string, not a rewrite.
- Status
- In development. No date announced — when there is one, it will be here.
- Self-hosted
- Unaffected. The engine, console and Helm chart stay Apache-2.0 and free, exactly as they are now.
Three ways in.
Smallest first.
The first one needs no cluster, no repository and no container runtime: one binary and a file. Start there; the other two are the same engine with more of it.
# one binary. no clone, no runtime. # grab your platform build from # github.com/lucheeseng827/dagron $ chmod +x dagron # run the graph below, live UI on :8787 $ ./dagron dev workflow.yaml
# published images — builds nothing $ git clone \ https://github.com/lucheeseng827/dagron $ cd dagron $ docker compose \ -f compose.quickstart.yaml up -d # console at /, API at /api — one port http://localhost:8080
# multi-arch images + OCI chart $ helm install dagron \ oci://registry-1.docker.io/mancube/dagron # or point it at your own Postgres $ helm install dagron \ oci://…/mancube/dagron \ --set postgres.enabled=false \ --set externalDatabaseSecret.name=\ dagron-db
The whole of workflow.yaml. Nothing is elided; this is a complete, valid spec.
name: finetune tasks: - name: prepare command: ["./prepare.sh"] - name: train needs: [prepare] command: ["./train.sh"] runner_class: gpu-a100 timeout_secs: 86400 max_attempts: 3 # retry gets DAGRON_RESUME_FROM - name: evaluate needs: [train] command: ["./eval.sh"]
Questions, answered.
What is dagron?
dagron is an open-source DAG workflow engine and scheduler. Define a workflow as a graph of tasks in YAML; dagron validates it and runs each task as soon as its dependencies succeed — concurrently, with retries and exponential backoff — from a single static Rust binary, using a database as its only state.
How is dagron different from Airflow?
dagron is a lightweight Airflow alternative: one Rust binary and plain YAML instead of a Python/Celery stack. No control plane, no cluster to operate — an embedded SQLite file works to start, and you switch to Postgres for multi-node. Tasks run as local subprocesses, Docker containers, or Kubernetes pods.
Is dagron free and open source?
Yes. The core engine, web UI, and Helm chart are Apache-2.0 and free to self-host indefinitely. An optional Enterprise edition adds SSO (SAML/OIDC/LDAP), a multi-tenant control plane, RBAC and audit, air-gapped install, and support.
How do I install dagron?
Install the Helm chart from Docker Hub (OCI): helm install dagron oci://registry-1.docker.io/mancube/dagron. To try it locally, docker compose -f compose.quickstart.yaml up -d pulls published images and builds nothing. See the get-started section.
What happens to a running task if the worker dies?
A running task holds a lease it renews every 10 seconds. If the worker stops renewing, the scheduler reclaims the task and retries it — and if that task reported a checkpoint, it is handed back as DAGRON_RESUME_FROM rather than starting over. A slow task is not a dead task: the lease is what tells them apart, which is why an eight-hour step is not reclaimed at minute fifty-nine. See the eight-hour step.
How hard is the port from Airflow?
The concepts map almost one to one — DAG to workflow, operator to a task with a command or
an image, sensor to type: wait, XCom to artifacts and datasets, pool to pool, and
the mapping table is on this page. What you give up is the Python DSL:
a dagron workflow is data, not a program, so your Python runs as a command, a container or a
pod while the graph stops being Python. There is no Airflow converter;
that port is by hand. Coming from Argo Workflows there is one — the open build ships an
importer that errors on what it cannot map rather than dropping it silently.
Can an AI agent drive dagron?
Yes. dagron-mcp is a Model Context Protocol server over stdio exposing 42 tools — register and update workflows, run them, long-poll to terminal, rerun, resubmit, inspect logs and artifacts — so an agent does the work without a human dropping to curl. It is the callee, not the agent: it holds no model and runs no loop. An agent conversation is expressed as an ordinary run, so each turn is a child run and state is artifacts rather than process memory.
What is not in the open build?
Multi-dataset composition and external dataset events, managed broker sources (Kafka, NATS, SQS, Redis), fleet management across many units, gang scheduling, SSO, RBAC with an audit trail, and envelope/BYOK-KMS encryption. The open build does not fail silently on any of them — it refuses with an error naming what is missing and the open path that does work. Everything in the eight-hour step is in the open build.
How does dagron make money, and what does that mean for the open build?
Open core. The engine, console, SDKs and Helm chart are Apache-2.0 and free to self-host indefinitely — that is the whole product for a single team, not a trial. Revenue comes from the Enterprise edition (the identity, tenancy and operations layer in the table above, self-hosted on your infrastructure) and, in future, dagron Cloud, a managed dagron with no announced date. The line between them is stated in the code itself: the open build names what it will not do rather than hiding it.