Skip to content

just_dna_enricher.ensembl

just_dna_enricher.ensembl

Live Ensembl resolution — for the rare rsIDs the frozen snapshot slice misses.

One backend: the REST variation API (rest.ensembl.org), a bare rsID → its GRCh38 mappings. Paced under Ensembl's published budget (PacingGate, once per attempt) and retried by tenacity on a transport error, a timeout, a 5xx or a 429, waiting at least what the answer's Retry-After asked for; a wait longer than net.RETRY_AFTER_CEILING is not slept, and nothing is asked until it passes. All network lives here (never in format/compiler). GRCh38 only.

There was a second, and it is gone (2026-11-04). A beta GraphQL leg (beta.ensembl.org/api/ graphql/variation, its query leeched from ensembl-mcp) was POSTed before every REST request. Probed by hand that day: the endpoint answers 301 to www.ensembl.org/api/graphql/variation; there the query's shape no longer validates (Cannot query field 'region_name' on type 'Location', HTTP 400); and a bare rsID is VARIANT_ID_NOT_FOUND even in a valid query, since that API wants a composite region:pos:rsid id. It could not answer a bare rsID on any day it was asked, so it cost a request per rsID and returned nothing. ensembl-graphql survives only as a resolution.csv source value a row written before 0.8 may carry.

EnsemblError

Bases: RuntimeError

A live Ensembl query failed on all backends (after retries + fallback).

RsidResolver

Bases: Protocol

What enrich(resolver=) requires of an injected resolver: resolve_rsid, and nothing else.

EnsemblResolver is one; a host may inject its own (a shared, paced one, or a test fake). Every method enrich calls beyond this one is an optional capability, checked against its own named Protocol before the call (RestWitness), so a resolver written against an older enricher keeps working and the capability it lacks is reported as not asked rather than raised as an AttributeError mid-run. The one optional attribute, non_nucleotide_rsids (RM272), is read with an empty default. close is called only on a resolver enrich built itself.

RestWitness

Bases: Protocol

The optional capability the placement witness asks (RM267): REST's own mapping of an rsID.

Added in 0.8, after hosts were already injecting resolvers. A resolver without it leaves Ensembl REST not asked for the run, counted, with one warning naming why.

EnsemblResolver dataclass

EnsemblResolver(
    settings: EnsemblSettings = EnsemblSettings(),
    _client: Client | None = None,
    _read_base: Callable[[str, int], str | None]
    | None = None,
    non_nucleotide_rsids: set[str] = set(),
    gate: PacingGate | None = None,
    _resume_at: float | None = None,
    _hold_lock: Lock = threading.Lock(),
)

Bases: Withholding

Resolve a bare rsID to its GRCh38 loci via live Ensembl REST.

resolve_rsid

resolve_rsid(
    rsid: str,
) -> tuple[list[dict] | None, str | None]

Return ([{chrom, start, ref, alts}, ...], source) for a bare rsID.

Asks REST (resolve_rsid_rest); the retired GraphQL leg is no longer asked first (module docstring). source is ensembl-rest — recorded per row so the manifest can report provenance.

Three outcomes, because two of them used to be one (S20). A non-empty list is an answer; [] is also an answer — Ensembl was reached and has no GRCh38 locus for this rsID — and None means Ensembl could not be asked at all, so its answer is unchecked rather than empty. Fusing the last two into ([], None) made a failed request read as a definite negative, and loci: [] plus "Ensembl has no locus" is exactly the fingerprint of a fabricated rsID: a consumer checking which ids in a machine-written document were real put two published variants in the fabricated pile on a flaky run. This is the tri-state rule the rest of the tree keeps — an unreachable source reports unknown, never the negative.

A 400 is an answer, not a failure: Ensembl 400s on rsIDs it cannot resolve (rs3216883, which dbSNP reports as merged). Every other status — a 429 throttle, a 403, a 404, a 410, a 5xx — and a transport error or a timeout return None (RM361: until then every 4xx was the empty answer, and a throttled run wrote not_found for live rsIDs). An empty answer now carries its source too, so a caller can record which link said nothing; that omission was the only trace the old code left, and it was a missing element in a set nobody diffs.

None with a source is the fourth outcome (RM268): Ensembl answered, and nothing it said could be placed. REST spells an insertion or a deletion with one side - at an interbase start, and such a locus is written only once the reference base before the event has been read and prefixed to every allele. When that base cannot be read, the locus is withheld, and an answer whose every locus was withheld is unchecked rather than empty, exactly like a request that failed; the source is returned so a caller can say which of the two happened.

Source code in enricher/src/just_dna_enricher/ensembl.py
def resolve_rsid(self, rsid: str) -> tuple[list[dict] | None, str | None]:
    """Return `([{chrom, start, ref, alts}, ...], source)` for a bare rsID.

    Asks REST (`resolve_rsid_rest`); the retired GraphQL leg is no longer asked first (module
    docstring). `source` is `ensembl-rest` — recorded per row so the manifest can report provenance.

    **Three outcomes, because two of them used to be one (S20).** A non-empty list is an answer;
    `[]` is *also* an answer — Ensembl was reached and has no GRCh38 locus for this rsID — and
    **`None` means Ensembl could not be asked at all**, so its answer is unchecked rather than
    empty. Fusing the last two into `([], None)` made a failed request read as a definite negative,
    and `loci: []` plus "Ensembl has no locus" is exactly the fingerprint of a fabricated rsID: a
    consumer checking which ids in a machine-written document were real put two published variants
    in the fabricated pile on a flaky run. This is the tri-state rule the rest of the tree keeps —
    an unreachable source reports unknown, never the negative.

    A **400 is an answer**, not a failure: Ensembl 400s on rsIDs it cannot resolve (`rs3216883`,
    which dbSNP reports as merged). Every other status — a 429 throttle, a 403, a 404, a 410, a
    5xx — and a transport error or a timeout return `None` (RM361: until then every 4xx was the
    empty answer, and a throttled run wrote `not_found` for live rsIDs). An empty answer now carries its source too, so a
    caller can record *which* link said nothing; that omission was the only trace the old code
    left, and it was a missing element in a set nobody diffs.

    **`None` with a source is the fourth outcome (RM268): Ensembl answered, and nothing it said
    could be placed.** REST spells an insertion or a deletion with one side `-` at an interbase
    `start`, and such a locus is written only once the reference base before the event has been
    read and prefixed to every allele. When that base cannot be read, the locus is withheld, and an
    answer whose every locus was withheld is unchecked rather than empty, exactly like a request
    that failed; the source is returned so a caller can say which of the two happened.
    """
    return self.resolve_rsid_rest(rsid)

resolve_rsid_rest

resolve_rsid_rest(
    rsid: str,
) -> tuple[list[dict] | None, str | None]

resolve_rsid's V1 half on its own: REST's /variation mapping, with the same four outcomes.

The placement witness (RM267) asks this, not resolve_rsid: its question is where REST's own mapping puts the rsID, through RM268's anchor, and an injected resolver's resolve_rsid may answer from any surface. It is an optional capability of an injected one (RestWitness).

Source code in enricher/src/just_dna_enricher/ensembl.py
def resolve_rsid_rest(self, rsid: str) -> tuple[list[dict] | None, str | None]:
    """`resolve_rsid`'s V1 half on its own: REST's `/variation` mapping, with the same four outcomes.

    The placement witness (RM267) asks this, not `resolve_rsid`: its question is where REST's own
    mapping puts the rsID, through RM268's anchor, and an injected resolver's `resolve_rsid` may
    answer from any surface. It is an optional capability of an injected one (`RestWitness`).
    """
    if self._held():
        logger.info("V1 REST for %s not asked: Ensembl's Retry-After has not passed", rsid)
        return None, None
    try:
        parsed = self._rest_rsid(rsid)
    except httpx.HTTPStatusError as exc:
        self._hold_for(exc)
        # Only a 400 is "no record" (RM361): a 5xx or a 429 after the retries, a 403, a 404 path or a
        # 410 retired route answered nothing about the rsID (`@unreachable-not-absent`).
        if exc.response.status_code != _REST_NO_RECORD_STATUS:
            self.note_retired("Ensembl REST", exc)
            logger.warning("V1 REST for %s failed: %s", rsid, exc)
            return None, None
        logger.info("V1 REST has no record for %s (%s)", rsid, exc)
        return [], "ensembl-rest"
    except (httpx.HTTPError, EnsemblError) as exc:
        logger.warning("V1 REST for %s could not be reached: %s", rsid, exc)
        return None, None
    return self._rest_answer(rsid, parsed)

data_release

data_release() -> str | None

The data release REST answers from, as a label (ensembl_116), or None when it will not say.

/info/data ({"releases": [116]}, read by hand on 2026-11-04), never /info/rest, which states the REST software's own version (15.12) and moves on a different clock from the data (RM373). Asked once by the run that used the live link, so the label is the release of that run's answers. A transport failure is retried; a status error, a body that is not JSON or one naming no single release is None with a log line: the rows are written, and only the release goes unrecorded.

Source code in enricher/src/just_dna_enricher/ensembl.py
def data_release(self) -> str | None:
    """The data release REST answers from, as a label (`ensembl_116`), or `None` when it will not say.

    `/info/data` (`{"releases": [116]}`, read by hand on 2026-11-04), never `/info/rest`, which states
    the REST software's own version (`15.12`) and moves on a different clock from the data (RM373).
    Asked once by the run that used the live link, so the label is the release of that run's answers.
    A transport failure is retried; a status error, a body that is not JSON or one naming no single
    release is `None` with a log line: the rows are written, and only the release goes unrecorded.
    """
    try:
        payload = self._rest_info_data()
    except (httpx.HTTPError, EnsemblError) as exc:
        logger.warning("Ensembl REST did not state its data release (%s); recorded as unknown.", exc)
        return None
    release = ensembl_data_release(payload)
    if release is None:
        logger.warning(
            "Ensembl REST's /info/data states no single release (%r); recorded as unknown.", payload
        )
        return None
    return f"{ENSEMBL_RELEASE_PREFIX}{release}"

ensembl_data_release

ensembl_data_release(payload: object) -> int | None

The one release /info/data states ({"releases": [116]}), or None when it states not exactly one.

Source code in enricher/src/just_dna_enricher/ensembl.py
def ensembl_data_release(payload: object) -> int | None:
    """The one release `/info/data` states (`{"releases": [116]}`), or `None` when it states not exactly one."""
    releases = payload.get("releases") if isinstance(payload, dict) else None
    if not isinstance(releases, list) or len(releases) != 1:
        return None
    (release,) = releases
    return release if isinstance(release, int) and not isinstance(release, bool) else None