Docs

CLI authoring

list, run, test, check, push — every command that operates on locally-registered pipelines.

The five commands you live in while building a pipeline.

esker list

Usage: esker list [OPTIONS]

No args, no flags. Lists registered pipelines. Empty stdout if nothing registered (silent zero — no "no pipelines found" message).

$ esker list
  global.spacex.rockets  1.0.0  weekly  2026-04-27 11:19
  us.sec.companies       2.0.0  daily   2026-05-03 19:00
  us.treasury.yields     2.0.0  daily   2026-05-03 19:01

Columns:

col source
domain REGISTRY key
version cls.SCHEMA.declared_version()
cadence getattr(cls, "cadence", "—")
last run mtime of ./output/<domain>.parquet, or never run

Notes:

  • last run reads from ./output/<domain>.parquet, not from any per-call --output. If you esker run -o data/, list will say never run.
  • Time is local-zone but unsuffixed. Other commands say UTC.

esker run

Usage: esker run [OPTIONS] DOMAIN_ID
  --verbose / -v
  --output  / -o   PATH   default: output

Execute a registered pipeline. Writes <DOMAIN_ID>.parquet and <DOMAIN_ID>.lineage.json to the output dir.

$ esker run us.treasury.yields
  us.treasury.yields@2.0.0
  4,205 records · 0.7s · output/us.treasury.yields.parquet

Owner-agnostic — the header uses <DOMAIN_ID>@<schema_version>, no owner prefix. Ownership is publish-time concern.

-v adds the full traceback after a blank line on error.

Exit codes:

  • 0 — success
  • 1 — domain not registered, or any exception in pipeline.run()

esker test

Usage: esker test [OPTIONS] [DOMAIN_ID]
  --update

Run fixtures. No network. DOMAIN_ID optional — omitted iterates every registered pipeline alphabetically.

$ esker test
  global.spacex.rockets@1.0.0
  2 passed · 0.0s

  us.sec.companies@2.0.0
  2 passed · 0.0s

  us.treasury.yields@2.0.0
  2 passed · 0.0s

On failure:

  us.sec.companies@2.0.0
  1 passed · 1 failed · 0.0s

  mismatch: short_cik
  --- expected
  +++ actual
  @@ -1,4 +1,4 @@
   {
  -  "cik": "0000320193",
  +  "cik": "320193",
     "ticker": "AAPL",
     "title": "Apple Inc."
   }

--update writes/overwrites expected_*.json files for missing or mismatched cases. Destructive — overwrites without prompting.

No fixtures dir → prints no fixtures (dim) and exits 0. Push, however, treats no-fixtures as a failure.

See Fixtures for the layout conventions and harness internals.

Exit codes:

  • 0 — every fixture passed (or --update succeeded)
  • 1 — any failure

esker check

Usage: esker check [OPTIONS] DOMAIN_ID
  --owner TEXT

Diff a registered pipeline's schema against the last published version. Read-only.

The output adapts to the case:

# First publish (no prior manifest)
  archie/us.treasury.yields
  2.0.0 · no prior version

# Same version already published
  archie/us.treasury.yields
  2.0.0 already published

# Major bump (skip-diff)
  archie/us.treasury.yields
  1.0.0 → 2.0.0 (major) · major bumps are not diffed

# Patch / minor diff, compatible
  archie/us.treasury.yields
  1.2.3 → 1.3.0 (minor) · compatible
  field 'name_length' added

# Patch / minor diff, incompatible (exits 1)
  archie/us.treasury.yields
  1.0.0 → 1.0.1 (patch) · incompatible
  field 'cik': pattern '^\d{10}$' → '^\d{8}$'
  required bump: major

--owner <handle> overrides the credentialed handle.

Without credentials and without --owner:

  not signed in — run 'esker login'

Exits 1 only on the incompatible case. Other cases (no prior version, already published, major skip, grandfather) exit 0.

For the full classification rules see Compatibility.

esker push

Usage: esker push [OPTIONS] DOMAIN_ID
  --owner            TEXT       (default: from credentials)
  --force-major
  --force-untested
  --verbose -v
  --output  -o       PATH       default: output

Run + upload. The most logic-heavy command.

$ esker push us.treasury.yields
  archie/us.treasury.yields@2.0.0
  4,205 records · 0.7s · output/us.treasury.yields.parquet
  pushed archie/us.treasury.yields@2.0.0

Three lines on success. The last is dim.

The full sequence, failure modes, and --force-major / --force-untested semantics live in Publishing.

See also

Edit on GitHub →View as Markdown