Public API

What `from esker import …` exports, with intent of each symbol.

The package's __init__.py is the public face. The convention: you should be able to from esker import X for any X in __all__ and not need to dive into submodules.

Exports

# Pipeline authoring
from esker import (
    pipeline,                              # decorator (80% case)
    EskerPipeline,                         # ABC for the explicit form
    EskerSource, Fetched,                  # source ABC + envelope
    BulkCsvSource, BulkJsonSource,         # built-in source impls
    EskerEntity, EskerID, EskerModel,      # protocol classes
    SemVer,                                # the semver type alias
    REGISTRY, register,                    # global dict + decorator
)

# Consuming
from esker import get, get_one, search

# Schema utilities
from esker import (
    DatasetManifest, Name,                 # manifest + name type alias
    OwnerHandle,                           # owner type alias
    LineageBatch, LineageBundle,           # lineage types
    CompatReport, check,                   # compat diff
    to_arrow_bytes, to_arrow_schema,       # JSON Schema → Arrow
    to_typescript,                         # JSON Schema → TS interface
)

# Testing
from esker import FixtureFailure, FixtureReport, run_fixtures

Grouped by use case

Authoring a pipeline (most common)

symbol when to reach for it
pipeline The decorator. 80% of pipelines.
register, REGISTRY Three-class form.
EskerPipeline, EskerSource, EskerModel Three-class form ABCs.
Fetched When implementing EskerSource.fetch_all() directly.
BulkJsonSource, BulkCsvSource Skip writing a source if the data fits one of these shapes.

See Pipelines, Records, Sources, Three-class form.

Consuming a published dataset

symbol use
get(ref) Returns a polars.DataFrame (or LazyFrame if lazy=True).
get_one(ref, esker_id=...) Single record by ID. Raises LookupError / ValueError.
search(ref, **criteria) Equality-filter. AND-composed across kwargs.

See Reading.

Working with the protocol primitives

symbol shape
EskerEntity The entity-resolver record. Currently unused by any path; deferred to a future release.
EskerID Annotated[str, Field(pattern=r"^esker:[a-z]{2,}:[a-z]+:[\w.-]+$")]
SemVer Annotated[str, Field(pattern=r"^\d+\.\d+\.\d+$")]
Name Annotated[str, Field(pattern=r"^[a-z0-9]+(\.[a-z0-9]+)+$")]
OwnerHandle Annotated[str, Field(pattern=r"^[a-z0-9](?:-?[a-z0-9])*$"), AfterValidator(_reject_reserved)], max 39
DatasetManifest The frozen Pydantic model that becomes manifest.json.
LineageBundle, LineageBatch The frozen models written as <domain>.lineage.json.
CompatReport, check Pure JSON Schema diff.
to_arrow_schema, to_arrow_bytes JSON Schema → Arrow.
to_typescript JSON Schema → TS interface.

Testing

symbol use
run_fixtures(cls, *, update=False) Programmatic fixture harness; returns FixtureReport.
FixtureReport, FixtureFailure Result types.

See Fixtures.

Not exported (intentional)

Some symbols exist but are submodule-only — using them isn't blessed, even though the import works.

  • DatasetRef — internal value object. Users pass strings; the SDK parses at the edge. from esker.schemas.ref import DatasetRef works but isn't public API.
  • bindings, hub, auth, compat (from esker.client) — CLI-side surfaces. Library users go through esker.get etc.
  • config — env-var accessors. Don't bake paths into your code; let the env handle it.
  • EskerPipeline.run() — the single supported path that mints published records. Don't call cls.published()(...) directly.

Two console scripts

pip install esker wires two console scripts pointing at the same CLI:

esker --help      # full name
esk --help        # short alias

Both invoke esker.cli:app. Use whichever you prefer.

Notable

  • The package has zero submodule re-exports below the top level. esker.client is a real package but from esker.client import get is not the documented path; from esker import get is.
  • esker.testing is a real submodule (the harness lives there) but run_fixtures etc. are also surfaced at the top level.

See also