just_dna_enricher.ensembl

just_dna_enricher.ensembl

Live Ensembl resolution — for the rare rsIDs the frozen snapshot slice misses.

Two backends, tried in order with a fallback the user asked for:

V2 — the beta GraphQL variation API (endpoint + query shape leeched from ensembl-mcp, with fastmcp/eliot dropped and stdlib logging in their place); and V1 — the legacy REST variation API (rest.ensembl.org), which is the workhorse for a bare rsID → coordinate (the beta GraphQL variant lookup needs a composite region:pos:rsid id, so a bare rsID falls through to REST). V1 also serves as the explicit fallback on 500/503 from V2.

tenacity retries each backend on transient transport/timeout errors; a 5xx is not retried but triggers the V2→V1 fallback. All network lives here (never in format/compiler). GRCh38 only.

EnsemblError

Bases: RuntimeError

A live Ensembl query failed on all backends (after retries + fallback).

EnsemblResolver dataclass

EnsemblResolver(
    settings: EnsemblSettings = EnsemblSettings(),
    _client: Client | None = None,
    _read_base: Callable[[str, int], str | None]
    | None = None,
)

Resolve a bare rsID to its GRCh38 loci via live Ensembl (V2 GraphQL → V1 REST fallback).

resolve_rsid

resolve_rsid(
    rsid: str,
) -> tuple[list[dict] | None, str | None]

Return ([{chrom, start, ref, alts}, ...], source) for a bare rsID.

Tries V2 GraphQL, then falls back to V1 REST on a 5xx (or when V2 yields nothing). source is ensembl-graphql or ensembl-rest — recorded per row so the manifest can report provenance.

Three outcomes, because two of them used to be one (S20). A non-empty list is an answer; [] is also an answer — Ensembl was reached and has no GRCh38 locus for this rsID — and None means Ensembl could not be asked at all, so its answer is unchecked rather than empty. Fusing the last two into ([], None) made a failed request read as a definite negative, and loci: [] plus "Ensembl has no locus" is exactly the fingerprint of a fabricated rsID: a consumer checking which ids in a machine-written document were real put two published variants in the fabricated pile on a flaky run. This is the tri-state rule the rest of the tree keeps — an unreachable source reports unknown, never the negative.

A 4xx is an answer, not a failure: Ensembl 400s on rsIDs it cannot resolve (rs3216883, which dbSNP reports as merged), so only a 5xx, a transport error or a timeout — the cases where nothing came back at all — return None. An empty answer now carries its source too, so a caller can record which link said nothing; that omission was the only trace the old code left, and it was a missing element in a set nobody diffs.

None with a source is the fourth outcome (RM268): Ensembl answered, and nothing it said could be placed. REST spells an insertion or a deletion with one side - at an interbase start, and such a locus is written only once the reference base before the event has been read and prefixed to every allele. When that base cannot be read, the locus is withheld, and an answer whose every locus was withheld is unchecked rather than empty, exactly like a request that failed; the source is returned so a caller can say which of the two happened.

Source code in enricher/src/just_dna_enricher/ensembl.py

def resolve_rsid(self, rsid: str) -> tuple[list[dict] | None, str | None]:
    """Return `([{chrom, start, ref, alts}, ...], source)` for a bare rsID.

    Tries V2 GraphQL, then falls back to V1 REST on a 5xx (or when V2 yields nothing). `source` is
    `ensembl-graphql` or `ensembl-rest` — recorded per row so the manifest can report provenance.

    **Three outcomes, because two of them used to be one (S20).** A non-empty list is an answer;
    `[]` is *also* an answer — Ensembl was reached and has no GRCh38 locus for this rsID — and
    **`None` means Ensembl could not be asked at all**, so its answer is unchecked rather than
    empty. Fusing the last two into `([], None)` made a failed request read as a definite negative,
    and `loci: []` plus "Ensembl has no locus" is exactly the fingerprint of a fabricated rsID: a
    consumer checking which ids in a machine-written document were real put two published variants
    in the fabricated pile on a flaky run. This is the tri-state rule the rest of the tree keeps —
    an unreachable source reports unknown, never the negative.

    A **4xx is an answer**, not a failure: Ensembl 400s on rsIDs it cannot resolve (`rs3216883`,
    which dbSNP reports as merged), so only a 5xx, a transport error or a timeout — the cases where
    nothing came back at all — return `None`. An empty answer now carries its source too, so a
    caller can record *which* link said nothing; that omission was the only trace the old code
    left, and it was a missing element in a set nobody diffs.

    **`None` with a source is the fourth outcome (RM268): Ensembl answered, and nothing it said
    could be placed.** REST spells an insertion or a deletion with one side `-` at an interbase
    `start`, and such a locus is written only once the reference base before the event has been
    read and prefixed to every allele. When that base cannot be read, the locus is withheld, and an
    answer whose every locus was withheld is unchecked rather than empty, exactly like a request
    that failed; the source is returned so a caller can say which of the two happened.
    """
    try:
        loci = self._graphql_rsid(rsid)
        if loci:
            return loci, "ensembl-graphql"
    except httpx.HTTPStatusError as exc:
        if exc.response.status_code not in _FALLBACK_STATUS:
            logger.warning("V2 GraphQL for %s failed (%s); trying REST", rsid, exc)
    except (EnsemblError, httpx.TransportError, httpx.TimeoutException) as exc:
        logger.warning("V2 GraphQL for %s errored (%s); trying REST", rsid, exc)

    try:
        loci, withheld, unplaceable = self._rest_rsid(rsid)
    except httpx.HTTPStatusError as exc:
        if exc.response.status_code in _FALLBACK_STATUS:
            logger.warning("V1 REST for %s failed: %s", rsid, exc)
            return None, None
        logger.info("V1 REST has no record for %s (%s)", rsid, exc)
        return [], "ensembl-rest"
    except (httpx.HTTPError, EnsemblError) as exc:
        logger.warning("V1 REST for %s could not be reached: %s", rsid, exc)
        return None, None
    if withheld:
        logger.info(
            "V1 REST for %s: %d one-sided indel locus/loci withheld, the anchor base could not be "
            "read (RM268)",
            rsid,
            withheld,
        )
        if not loci:
            return None, "ensembl-rest"
    if unplaceable:
        # RM271. A mapping with no nucleotide allele (`dbSNP_novariation`, an `N` run) is not a
        # locus. Permanent, unlike an unreadable anchor, so an rsID left with none is the existing
        # answered-empty `[]`, and the words say it is Ensembl's own "no variation here".
        log = logger.warning if not loci else logger.info
        log(
            "V1 REST for %s: %d mapping(s) carry no nucleotide allele (e.g. dbSNP_novariation) and "
            "are not loci (RM271)%s",
            rsid,
            unplaceable,
            "; no placeable locus is left, so it is written as not found" if not loci else "",
        )
    return loci, "ensembl-rest"