Public API
What `from esker import …` exports, with intent of each symbol.
The package's __init__.py is the public face. The convention: you should be able to from esker import X for any X in __all__ and not need to dive into submodules.
Exports
# Pipeline authoring
from esker import (
pipeline, # decorator (80% case)
EskerPipeline, # ABC for the explicit form
EskerSource, Fetched, # source ABC + envelope
BulkCsvSource, BulkJsonSource, # built-in source impls
EskerEntity, EskerID, EskerModel, # protocol classes
SemVer, # the semver type alias
REGISTRY, register, # global dict + decorator
)
# Consuming
from esker import get, get_one, search
# Schema utilities
from esker import (
DatasetManifest, Name, # manifest + name type alias
OwnerHandle, # owner type alias
LineageBatch, LineageBundle, # lineage types
CompatReport, check, # compat diff
to_arrow_bytes, to_arrow_schema, # JSON Schema → Arrow
to_typescript, # JSON Schema → TS interface
)
# Testing
from esker import FixtureFailure, FixtureReport, run_fixtures
Grouped by use case
Authoring a pipeline (most common)
| symbol | when to reach for it |
|---|---|
pipeline |
The decorator. 80% of pipelines. |
register, REGISTRY |
Three-class form. |
EskerPipeline, EskerSource, EskerModel |
Three-class form ABCs. |
Fetched |
When implementing EskerSource.fetch_all() directly. |
BulkJsonSource, BulkCsvSource |
Skip writing a source if the data fits one of these shapes. |
See Pipelines, Records, Sources, Three-class form.
Consuming a published dataset
| symbol | use |
|---|---|
get(ref) |
Returns a polars.DataFrame (or LazyFrame if lazy=True). |
get_one(ref, esker_id=...) |
Single record by ID. Raises LookupError / ValueError. |
search(ref, **criteria) |
Equality-filter. AND-composed across kwargs. |
See Reading.
Working with the protocol primitives
| symbol | shape |
|---|---|
EskerEntity |
The entity-resolver record. Currently unused by any path; deferred to a future release. |
EskerID |
Annotated[str, Field(pattern=r"^esker:[a-z]{2,}:[a-z]+:[\w.-]+$")] |
SemVer |
Annotated[str, Field(pattern=r"^\d+\.\d+\.\d+$")] |
Name |
Annotated[str, Field(pattern=r"^[a-z0-9]+(\.[a-z0-9]+)+$")] |
OwnerHandle |
Annotated[str, Field(pattern=r"^[a-z0-9](?:-?[a-z0-9])*$"), AfterValidator(_reject_reserved)], max 39 |
DatasetManifest |
The frozen Pydantic model that becomes manifest.json. |
LineageBundle, LineageBatch |
The frozen models written as <domain>.lineage.json. |
CompatReport, check |
Pure JSON Schema diff. |
to_arrow_schema, to_arrow_bytes |
JSON Schema → Arrow. |
to_typescript |
JSON Schema → TS interface. |
Testing
| symbol | use |
|---|---|
run_fixtures(cls, *, update=False) |
Programmatic fixture harness; returns FixtureReport. |
FixtureReport, FixtureFailure |
Result types. |
See Fixtures.
Not exported (intentional)
Some symbols exist but are submodule-only — using them isn't blessed, even though the import works.
DatasetRef— internal value object. Users pass strings; the SDK parses at the edge.from esker.schemas.ref import DatasetRefworks but isn't public API.bindings,hub,auth,compat(fromesker.client) — CLI-side surfaces. Library users go throughesker.getetc.config— env-var accessors. Don't bake paths into your code; let the env handle it.EskerPipeline.run()— the single supported path that mints published records. Don't callcls.published()(...)directly.
Two console scripts
pip install esker wires two console scripts pointing at the same CLI:
esker --help # full name
esk --help # short alias
Both invoke esker.cli:app. Use whichever you prefer.
Notable
- The package has zero submodule re-exports below the top level.
esker.clientis a real package butfrom esker.client import getis not the documented path;from esker import getis. esker.testingis a real submodule (the harness lives there) butrun_fixturesetc. are also surfaced at the top level.