dagron/ docs
Reference

CLI & configuration

Every binary, positional argument, environment variable, and Cargo feature dagron reads. dagron has no CLI flag parser — this page is the whole surface.

The shape of the CLI

dagron is configured in three layers, and only three:

There is no run subcommand: the first positional is the DAG path. The two companion tools that do take flags (dagron-plan, dagron-import) are separate binaries, documented below.

dagron — run the engine

INVOCATION
dagron [dev] [DAG_PATH] [DB_TARGET]
dagron validate <file|dir>... [--json]
dagron archive-compact [DB_TARGET]
ArgumentDefaultMeaning
devZero-infra local quickstart: SQLite plus the management API and Swagger UI on 127.0.0.1:8787 (sets API_ADDR if unset), and the process stays resident. With no DAG file present it starts idle and waits for POST /runs. Requires the default ops feature.
DAG_PATHexamples/simple_dag.yamlWorkflow YAML for the file source. First positional — second under dagron dev.
DB_TARGETworkflow.db (sqlite build) / $DATABASE_URL, then postgres://localhost/workflow (postgres build)SQLite file path, or Postgres connection string. Second positional — third under dagron dev.

dev consumes the first positional, so the others shift right: dagron dev [DAG_PATH] [DB_TARGET]. dagron dev foo.db therefore reads foo.db as the workflow and still writes state to workflow.db. The datastore is the third token (dagron dev my.yaml my.db). The startup line prints the datastore actually in use — check it when a run seems to vanish.

One-shot vs. resident

With only a DAG path, dagron executes that one workflow and exits when all runs drain:

ONE-SHOT RUN
$ ./target/release/dagron examples/simple_dag.yaml
INFO dagron_engine: run complete run_id=… status=succeeded
INFO dagron_engine: all runs drained — scheduler exiting

The process stays resident instead when anything long-lived is enabled: dagron dev, an API_ADDR (the ops API), CRON_CONFIG, GC_RETENTION_SECS, or DB_SCHEDULES. In dev mode, submit a run with raw workflow YAML as the body:

DEV MODE
$ cargo build --release -p dagron
$ ./target/release/dagron dev
INFO dagron_engine: dagron dev — local quickstart: datastore workflow.db, management API + Swagger UI on http://127.0.0.1:8787/docs (override with API_ADDR)
$ curl -s -X POST localhost:8787/runs --data-binary @examples/simple_dag.yaml
# {"run_id":"…"}

The full stack with the web UI is one command: docker compose up --build (or podman compose up --build).

Two HTTP surfaces

dagron exposes two HTTP APIs with a deliberate boundary. The engine ops API (built with the default ops feature, bound at API_ADDR) is unauthenticated and meant to stay cluster-internal; it is self-describing — OpenAPI 3 at /openapi.yaml and /openapi.json, Swagger UI at /docs. The dagron-api UI edge is the JWT-gated surface on PORT for browsers, automation, and agents. Full endpoint reference: API.md on GitHub.

dagron validate and dagron archive-compact

dagron validate

dagron validate <file|dir>... [--json] is an offline spec lint: it parses, template-expands, and graph-validates each *.yaml/*.yml through the same pipeline every submit path uses. Directories are walked recursively (hidden directories skipped). No datastore, no server — it works in every build, so it fits pre-commit hooks and CI.

LINT IN CI
$ dagron validate workflows/ --json

dagron archive-compact

dagron archive-compact [DB_TARGET] runs one bounded sweep that folds archived run-*.json documents into the date-partitioned Parquet dataset (compact/tasks/dt=…/) — the Kubernetes CronJob shape. It reads the archive sink env (GC_ARCHIVE_DIR / GC_ARCHIVE_URL) and optionally a datastore to stamp archived_runs. Documents younger than GC_ARCHIVE_COMPACT_MIN_AGE_DAYS (default 30) stay individually retrievable; 0 compacts everything eligible.

dagron archive-compact requires building with --features archive-parquet. Without it the subcommand is a clear startup error, never a silent no-op. Combine with archive-s3 / archive-gcs / archive-azure to compact a cloud archive.

Companion binaries

BinaryWhat it is
dagron-apiThe authenticated UI edge: JWT sessions, the REST/SSE API the console and automation use. Postgres-only, listens on PORT.
dagron-mcpModel Context Protocol server over stdio — lets an AI agent drive workflows through dagron-api.
dagron-importConvert another orchestrator's workflow to dagron DAG YAML. Today: Argo Workflows.
dagron-planDiff two workflow specs through the real parser — a PR-ready plan of what a change does to the resolved DAG.
dagron-gitopsPolling worker that reconciles workflow definitions from Git repositories into the datastore.

dagron-api

Stateless and Postgres-only (SSE needs LISTEN/NOTIFY). It takes no arguments; configuration is environment-only — see the variables below. DATABASE_URL and DAGRON_JWT_SECRET are required at startup.

dagron-mcp

No arguments; JSON-RPC 2.0 over stdio, logs to stderr so stdout carries only protocol messages. Configured by two variables: DAGRON_API_URL (default http://localhost:8080) and DAGRON_MCP_TOKEN (a session JWT sent as Authorization: Bearer). The tool catalogue is below.

dagron-import

dagron-import argo <workflow.yaml> — exactly two arguments; only the argo importer exists. It prints a dagron DAG YAML to stdout; pipe it into a file or POST /api/runs to migrate. An unknown importer or wrong argument count exits 2.

MIGRATE AN ARGO WORKFLOW
$ dagron-import argo workflow.yaml > my_dag.yaml

dagron-plan

Both specs are resolved through the real dagron parser and expander, so the plan reflects what would actually run. Output is GitHub-flavored markdown (summary, per-task diff, and a Mermaid graph of the resulting DAG) — pipe it into a PR comment.

USAGE
dagron-plan <base.yaml> <head.yaml>         # diff two files
dagron-plan --git <base>..<head> <path>     # diff a file across two refs
dagron-plan --git <base> <path>             # diff <base>:<path> vs the worktree

dagron-gitops

A polling worker with no ports: it scans registered repositories for YAML files carrying a tasks: key, validates each with the engine's own parser, and upserts it as a workflow definition. Repositories are connected from the console or the API; the reconcile is idempotent. It is the one dagron image that carries a git binary, which is why it ships separately — deploy it only if you use GitOps. Configuration: DATABASE_URL (required), GITOPS_POLL_SECS (default 60), DAGRON_GIT_TOKEN (sent only to trusted forge hosts over HTTPS), RUST_LOG.

Environment variables

The tables below are the user-facing subset. The complete list — streaming knobs, split-DSN seams, KMS envelope encryption, CA trust, and per-variable detail — is CONFIG.md on GitHub.

Engine core

VariableDefaultWhat it does
EXECUTORlocalTask backend: local, docker, or kubernetes/k8s. Unrecognized values warn and fall back to local; kubernetes without its Cargo feature is a startup error.
WORKER_COUNT16Worker-pool size — the max number of concurrently running tasks (min 1).
SOURCEfilefile (one-shot DAG file) or stream (follow an NDJSON event file or named pipe at STREAM_PATH). Managed broker connectors (redis/sqs/kafka/nats/events) Enterprise error at startup here with a pointer.
MAX_INFLIGHT_RUNS64Admission valve: cap on simultaneously active runs; overflow stays buffered at the source. The ops API answers 429 with Retry-After above it (0 disables the API-side cap).
DATABASE_URLpostgres://localhost/workflowPostgres builds only; the positional DB_TARGET wins. Redacted before logging.
DB_MAX_CONNECTIONS8Postgres pool size (min 2). SQLite ignores it (pinned to 1 by design).
DOCKER_IMAGEalpine:latestDefault image for EXECUTOR=docker (also the Kubernetes fallback).
K8S_IMAGE$DOCKER_IMAGEImage for the Kubernetes executor.
K8S_NAMESPACEdefaultKubernetes executor namespace.
RUNNER_CLASSESunset = claim every classRestrict this scheduler to claiming tasks whose runner_class is in the comma list (e.g. etl,pulse). A typo is a startup error, not an unclaimable task class.
POOLSunset = no poolsNamed concurrency pools, name:slots comma list (e.g. POOLS=etl:4,db:2). Keep the value identical across HA replicas.

Ops API, scheduling, and GC

VariableDefaultWhat it does
API_ADDRunset = ops API disabledBind address of the engine's unauthenticated ops API; also keeps the process resident. dagron dev sets 127.0.0.1:8787.
CRON_CONFIGunset = cron offPath to the cron schedule YAML. Leadership-gated; keeps the process resident.
DB_SCHEDULESoff1/true: fire the DB-backed schedules that dagron-api manages. Leadership-gated; resident.
LEADER_LEASE_SECS30Leadership lease for cron/GC/schedules (exactly-one-node guarantee).
GC_RETENTION_SECSunset = GC offRetention window for the run/task GC. Leadership-gated; resident.
GC_INTERVAL_SECS3600GC sweep interval.
WAIT_POLL_SECS15Poll interval for type: wait HTTP sensors; a parked sensor is polled at most once per interval and succeeds on the first 2xx.

Artifacts and archive

VariableDefaultWhat it does
GC_ARCHIVE_DIRunset = plain purgeArchive-before-purge: the GC exports each expired terminal run as a self-contained JSON document and purges only verified exports.
GC_ARCHIVE_URLunsetCloud archive-before-purge: s3://, gs://, az://, or azure:// bucket/prefix. Requires the matching Cargo feature — a scheme without its feature is a startup error, never a silent plain purge. Wins over GC_ARCHIVE_DIR. Credentials come from the backend's standard env (AWS_* — including AWS_ENDPOINT_URL for MinIO — / GOOGLE_* / AZURE_*).
GC_ARCHIVE_COMPACT_MIN_AGE_DAYS30dagron archive-compact only: younger documents stay individually retrievable; older ones fold into Parquet and become analytics-only.
DAGRON_ARTIFACT_DIRunset = offLocal artifact store root; each task gets its run's shared dir injected as DAGRON_ARTIFACTS, plus a per-task DAGRON_CHECKPOINT_DIR for checkpoint-aware resume.
DAGRON_ARTIFACT_URLunset = offCloud artifact/checkpoint location (s3://, gs://, az://). The engine injects per-run/per-task URLs (DAGRON_ARTIFACTS_URL, DAGRON_CHECKPOINT_URL) into tasks.

Secrets and masking

VariableDefaultWhat it does
DAGRON_SECRET_<NAME>unsetValue for a task env value_from: { secret: <name> } reference. Resolved at dispatch; masked in output.
DAGRON_SECRETS_DIRunsetDirectory of secret files (one per secret, filename = secret name) — the SOPS / External-Secrets / Kubernetes-secret mount convention. Checked after DAGRON_SECRET_<NAME>.
DAGRON_ENV_SECRET_KEYunset = env-secret store offAES-256-GCM key for UI-managed environment secrets. Must be set identically on both dagron-api (encrypts on write) and the engine (decrypts at dispatch).
DAGRON_SENSITIVE_ENV_PATTERNSSECRET,TOKEN,PASSWORD,PASSWD,PWD,CREDENTIAL,APIKEY,ACCESS_KEY,PRIVATE_KEYTask env var name substrings (case-insensitive) whose values are masked to *** in task output and logs. Set empty to disable name-based masking.
DAGRON_REDACT_ENVunsetEngine-process env var names whose values are always masked in task output (e.g. DATABASE_URL), on top of the pattern matching above.

dagron-api (UI edge)

VariableDefaultWhat it does
DATABASE_URLrequired (startup error)The same Postgres database the engine writes.
DAGRON_JWT_SECRETrequired (startup error)HS256 key that signs and validates session JWTs. Must be at least 32 characters.
PORT8080Listen port.
DAGRON_COOKIE_SECUREtrueSecure flag on the dagron_session cookie. Set false only for plain-HTTP local dev.
DAGRON_SESSION_TTL_SECS604800Session/JWT lifetime (7 days).
DAGRON_ADMIN_EMAIL / DAGRON_ADMIN_PASSWORDunset = no bootstrapIdempotently seed a first admin at startup; never resets an existing user. Password must be at least 8 characters.
GITHUB_TOKEN / GIT_REPOunset = sync answers 501Enable workflow-to-Git PR sync. GIT_BASE (default main), GIT_PATH_PREFIX (default dags/), and GIT_API_BASE (default https://api.github.com) tune it.

Logging (all binaries)

Every binary shares the same logging knobs: RUST_LOG (full tracing filter, wins over everything), LOG_LEVEL (default info), and LOG_FORMAT (full, compact, pretty, or json; default full). Further formatting toggles are in CONFIG.md.

Cargo features

Features are compile-time: a value that names a subsystem the binary was not built with fails at startup with a message, never a silent downgrade.

FeatureDefaultEffect
sqliteyesEmbedded single-writer SQLite datastore. Exactly one of sqlite/postgres must be enabled — both or neither is a compile error.
postgresnoPostgres datastore: LISTEN/NOTIFY wake, multi-worker claim. Required by HA and by the UI stack.
opsyesThe engine management API (API_ADDR), cron, retention GC, DB schedules, leadership.
kubernetesnoEXECUTOR=kubernetes. Without it that value is a startup error.
otelnoOpenTelemetry: TRACEPARENT injection into dispatched tasks, and OTLP (HTTP/protobuf) span export when OTEL_EXPORTER_OTLP_ENDPOINT is set.
archive-s3noS3 archive sink (GC_ARCHIVE_URL=s3://…, including MinIO/Ceph via AWS_ENDPOINT_URL). Implies ops.
archive-gcsnoGoogle Cloud Storage archive sink (GC_ARCHIVE_URL=gs://…). Implies ops.
archive-azurenoAzure Blob Storage archive sink (GC_ARCHIVE_URL=az://… or azure://…). Implies ops.
archive-parquetnoEnables dagron archive-compact. Heavy (arrow + parquet), hence its own feature. Implies ops.
enterprisenoEnterprise scheduler capabilities such as gang co-scheduling (RUNNER_GANGS); inert on an open build.

MCP tools

dagron-mcp exposes nine tools over stdio. It talks to the JWT-gated dagron-api edge — never the engine's internal ops API — so every access control on the UI edge applies identically to agents.

ToolArgumentsBacked by
dagron_list_runsGET /api/runs
dagron_get_runrun_idGET /api/runs/{id}
dagron_submit_runyamlPOST /api/runs
dagron_cancel_runrun_idPOST /api/runs/{id}/cancel
dagron_get_task_logsrun_id, task_id, + log filterGET /api/runs/{id}/tasks/{tid}/logs
dagron_get_run_logsrun_id, task, + log filterGET /api/runs/{id}/logs
dagron_get_metricsGET /api/metrics
dagron_list_dead_letterslimit (1–500, default 100)GET /api/dead-letters?limit=
dagron_get_run_eventsrun_id, wait_ms (100–10000, default 2000)bounded read of GET /api/runs/{id}/stream (SSE)

Reach for dagron_get_run_logs first when a run failed: one call returns every task's output as a single attributed, server-filtered stream. Both log tools take the same filter arguments — q, exclude, regex, level, case, context, limit, tail.

Register the server with an MCP client — the binary, or the published mancube/dagron-mcp image over docker run -i:

MCP CLIENT CONFIG
{
  "mcpServers": {
    "dagron": {
      "command": "dagron-mcp",
      "env": {
        "DAGRON_API_URL": "http://localhost:8080",
        "DAGRON_MCP_TOKEN": "<session-jwt>"
      }
    }
  }
}

The transport is newline-delimited JSON-RPC 2.0 on stdio. Security guidance — token scoping, edge isolation, executor sandboxing — is in MCP.md on GitHub.