Skip to content

just_dna_enricher.net

just_dna_enricher.net

Shared HTTP-politeness primitives for the network tier.

Extracted from gnomad.py when a second and third rate-limited service arrived (NCBI eutils, the PMC ID converter, Europe PMC, OLS4, HGNC). Nothing here knows about any particular API — it is the pacing, batching and ordering discipline every client in this package has to obey, in one place so the rule cannot drift between them.

Two of these look trivial and are not:

  • PacingGate takes its clock and its sleep as parameters, so a test can prove a six-second interval is honoured without a suite that really sleeps six seconds per request.
  • dedupe preserves first-occurrence order rather than going through a set, because the order requests are made in decides the order rows are emitted in, and emitted order is part of artifact.digest (Principle 7).

PacingGate dataclass

PacingGate(
    interval: float,
    clock: Callable[[], float] = time.monotonic,
    sleeper: Callable[[float], None] = time.sleep,
    last: float | None = None,
    spent: int = 0,
    _lock: Lock = threading.Lock(),
)

Enforce a minimum interval between requests, on an injectable clock.

The clock and the sleep are parameters rather than direct time calls so a test can prove the gate honours the interval without a suite that really sleeps six seconds per request. Monotonic by default, so a wall-clock adjustment mid-run cannot collapse the interval to zero.

One gate is safe to share across threads, and it had to become so (S15). LookupClients' docstring tells callers to hold a client and reuse it — precisely because a fresh one per question would discard this state — so a server running its blocking work through a thread pool shares one gate by following our own advice. The unsynchronized version read last, slept, then wrote it, so two threads could both find the interval elapsed, both skip the sleep, and turn a published 3/s budget into 6/s. That budget is a courtesy someone else enforces by blocking the operator's IP, so "single-threaded callers only" was not a contract worth keeping unstated or worth keeping.

The lock covers the bookkeeping, not the sleep. Each caller reserves the next free slot and then waits for it alone, so N callers get N slots spaced interval apart rather than serializing behind one lock held across a sleep — same guarantee, and no thread is blocked by another's wait. Behaviour on a single thread is unchanged.

spent is what the gate admitted, and the one number a host cannot otherwise get (S95). A proxy metering egress per upstream had to charge by the shape of a request — an upper bound, because nothing downstream reported the calls actually made. Every egressing client waits on its gate once per attempt, inside its retry loop (gnomad._post, eutils._request), so one increment is one upstream attempt: a 429 retried three times counts three, and a snapshot hit that never reached the gate counts nothing. Monotonic, bumped under the same lock as the slot, and never reset — a reader that wants a rate takes two readings.

attempt_floor

attempt_floor(default: int)

Bases: stop_base

stop_after_attempt(default), resolved per call so a deployment can raise it (RM42).

Drop-in for the stop_after_attempt(n) it replaces, and deliberately only for a bare one: a composed policy (stop_after_attempt(3) | stop_after_delay(60)) means both, and raising one term silently changes something whose author meant the conjunction. None of this tier's policies is composed today; the rule matters the day one is.

Deliberately no count in this prose. It read "the nine policies" in two places while the tree carried twelve, and nobody noticed because the guard was a floor (len(found) >= 9) walking seven of the nine modules that own one -- so three new policies and two whole unwalked modules were both invisible. @registry-completeness, the same shape as RM96's _ALL_MODELS hole: a number in prose is a registry nothing iterates. test_gated_snapshots.py now discovers the modules instead of listing them and asserts an equality over what it walked, which is a claim that cannot go stale.

Source code in enricher/src/just_dna_enricher/net.py
def __init__(self, default: int) -> None:
    self.default = default

EndpointRetired

Bases: Exception

Mixed into every client's *Retired error: the source answered HTTP 410 (RM361).

A marker, not a catchable family of its own: handlers keep catching their client's could-not-ask type, and read the reason off the exception with could_not_ask_reason.

Withholding

Mixed into every client whose contract is a withheld value rather than an exception (RM361).

Such a client answers a failed request with None (or its outcome's could-not-ask arm), so a 410 reached its caller as an outage. retired is how it says otherwise without raising: the retired_message of the first 410 the client met, None until then. Sticky for the client's life, because a retired route is a fact about the source rather than about one request; a caller recording why a question went unanswered reads it with withheld_reason.

A run, though, is not the client's life: a caller may reuse one client across runs (RM404's LookupClients), and run 1's 410 must not tell run 2's timeout to upgrade (RM446). So every 410 is also counted in retired_met, which a run reads before it asks (retired_mark) and compares after (withheld_reason(client, since=mark)); the client itself is never reset, since a caller's client is not the run's to alter.

note_retired

note_retired(
    source: str, failure: BaseException | Response
) -> bool

Record a 410 from failure (an exception or a response); True when it was one.

Source code in enricher/src/just_dna_enricher/net.py
def note_retired(self, source: str, failure: BaseException | httpx.Response) -> bool:
    """Record a 410 from `failure` (an exception or a response); `True` when it was one."""
    response = failure.response if isinstance(failure, httpx.HTTPStatusError) else failure
    if not isinstance(response, httpx.Response) or response.status_code != GONE:
        return False
    self.retired_met += 1
    if self.retired is None:
        self.retired = retired_message(source, response)
        logger.warning("%s", self.retired)
    return True

StreamedFile dataclass

StreamedFile(
    path: Path,
    sha256: str,
    etag: str | None = None,
    last_modified: str | None = None,
)

What one bulk download established: where it landed, its digest, and the source's own labels.

The digest is returned rather than logged. Four of the eleven copies computed a sha256 while streaming and then only wrote it to the log, so a caller that needed to record the provenance of the bytes it had just fetched had to hash the file again (@dont-discard-computed). Two of them later grew a tuple[Path, str] return for exactly that reason, one lane at a time.

etag and last_modified are the source's, None when it sends neither. They are how a lane can later ask has this file changed without downloading it again — MANE, CIViC and PubMind record them today, and the rest get them for free rather than growing their own copy later.

wait_retry_after

wait_retry_after(
    fallback: wait_base,
    ceiling: float = RETRY_AFTER_CEILING,
)

Bases: wait_base

A tenacity wait that is at least what the server's Retry-After asked for, else fallback's.

Never shorter than the fallback, so a server asking for less than the backoff does not shorten it; capped at ceiling, which retry_if_transient_within keeps a retry from ever reaching.

Source code in enricher/src/just_dna_enricher/net.py
def __init__(self, fallback: wait_base, ceiling: float = RETRY_AFTER_CEILING) -> None:
    self.fallback = fallback
    self.ceiling = ceiling

check_batch_size

check_batch_size(owner: str, size: int) -> None

Raise ValueError for a batch_size below 1, naming owner's field (RM455).

batched with a size of 0 yields no batch, so a client asked nothing and answered {}, which every caller reads as "no record for any id". Each settings class calls this when it is built and again where its batched read reads the field, since a dataclass field may be set after construction (the form 0.9 gave EnsemblSettings, review round 1 item 8).

Source code in enricher/src/just_dna_enricher/net.py
def check_batch_size(owner: str, size: int) -> None:
    """Raise `ValueError` for a `batch_size` below 1, naming `owner`'s field (RM455).

    `batched` with a size of 0 yields no batch, so a client asked nothing and answered `{}`, which every
    caller reads as "no record for any id". Each settings class calls this when it is built and again
    where its batched read reads the field, since a dataclass field may be set after construction (the
    form 0.9 gave `EnsemblSettings`, review round 1 item 8).
    """
    if size < 1:
        raise ValueError(
            f"{owner}.batch_size={size} is below 1: no batch would be asked, so nothing would be answered"
        )

batched

batched(items: list[T], size: int) -> Iterator[list[T]]

Split into batches of at most size, preserving order (so emission stays deterministic).

Source code in enricher/src/just_dna_enricher/net.py
def batched[T](items: list[T], size: int) -> Iterator[list[T]]:
    """Split into batches of at most `size`, preserving order (so emission stays deterministic)."""
    iterator = iter(items)
    while batch := list(itertools.islice(iterator, size)):
        yield batch

dedupe

dedupe(items: Iterable[T]) -> list[T]

First-occurrence-order de-duplication (Principle 7: never set iteration for emitted order).

Source code in enricher/src/just_dna_enricher/net.py
def dedupe[T](items: Iterable[T]) -> list[T]:
    """First-occurrence-order de-duplication (Principle 7: never `set` iteration for emitted order)."""
    seen: set[T] = set()
    out: list[T] = []
    for item in items:
        if item not in seen:
            seen.add(item)
            out.append(item)
    return out

retry_attempts

retry_attempts(default: int) -> int

How many attempts this client may make: its own default, raised to the configured floor.

A floor, never a flat setting. The per-client numbers are deliberate — gnomAD and eutils sit at 4 because their budgets are the tightest — so a single value that set every client would flatten tuning that was chosen on purpose, while one that raises preserves it. Below the default it is a no-op rather than a way to make a client give up sooner; there is no deployment that wants less persistence than an author at a terminal, and allowing it would turn one variable into a footgun.

Why a knob exists at all: three attempts is right for the audience the CLI was written for — a person who would rather see a failure in ten seconds than wait out a flapping upstream. It is wrong for the other shape the 0.5 tiering created, a server running enrich() inside an unattended publish, where giving up on a transient 502 does not cost ten seconds, it costs the publisher a whole re-upload of a module the server had already accepted and validated. Two callers wanting opposite things from one constant is the definition of a knob.

Safe to raise because every gated client with a published rate budget paces before it retries: an extra attempt spends a slot of that budget rather than bursting past it. The AlphaGenome Atlas client's gate has a zero interval, because the Atlas publishes no budget and none was found up to 21 calls/s (RM307); it still counts every attempt.

Source code in enricher/src/just_dna_enricher/net.py
def retry_attempts(default: int) -> int:
    """How many attempts this client may make: its own default, **raised** to the configured floor.

    **A floor, never a flat setting.** The per-client numbers are deliberate — gnomAD and eutils sit at
    4 because their budgets are the tightest — so a single value that *set* every client would flatten
    tuning that was chosen on purpose, while one that *raises* preserves it. Below the default it is a
    no-op rather than a way to make a client give up sooner; there is no deployment that wants less
    persistence than an author at a terminal, and allowing it would turn one variable into a footgun.

    Why a knob exists at all: three attempts is right for the audience the CLI was written for — a
    person who would rather see a failure in ten seconds than wait out a flapping upstream. It is wrong
    for the other shape the 0.5 tiering created, a **server** running `enrich()` inside an unattended
    publish, where giving up on a transient 502 does not cost ten seconds, it costs the publisher a
    whole re-upload of a module the server had already accepted and validated. Two callers wanting
    opposite things from one constant is the definition of a knob.

    Safe to raise because every gated client with a published rate budget **paces before it retries**:
    an extra attempt spends a slot of that budget rather than bursting past it. The AlphaGenome Atlas
    client's gate has a zero interval, because the Atlas publishes no budget and none was found up to
    21 calls/s (RM307); it still counts every attempt.
    """
    global _env_loaded, _file_attempts
    if not _env_loaded:
        # Imported here rather than at module scope: `locations` is a leaf and `net` is a leaf, and
        # making one import the other for one call would couple them permanently. This is the guarded
        # exception the house rule allows for exactly this shape.
        from just_dna_enricher.locations import env_value

        _file_attempts = env_value(RETRY_ATTEMPTS_ENV)
        _env_loaded = True
    stated = os.environ.get(RETRY_ATTEMPTS_ENV, _file_attempts)
    raw = (stated or "").strip()
    if not raw:
        return default
    try:
        configured = int(raw)
    except ValueError:
        logger.warning(
            "%s=%r is not an integer; using this client's own %d attempt(s).",
            RETRY_ATTEMPTS_ENV,
            raw,
            default,
        )
        return default
    return max(default, configured)

is_transient_status

is_transient_status(exc: BaseException) -> bool

Whether exc is an HTTP status a second attempt could change (a 5xx or a 429).

Source code in enricher/src/just_dna_enricher/net.py
def is_transient_status(exc: BaseException) -> bool:
    """Whether `exc` is an HTTP status a second attempt could change (a 5xx or a 429)."""
    if not isinstance(exc, httpx.HTTPStatusError):
        return False
    status = exc.response.status_code
    return status >= TRANSIENT_STATUS_FLOOR or status == THROTTLED

retry_if_transient

retry_if_transient(
    *also: type[BaseException],
) -> retry_base

Retry a transport failure, a timeout, a 5xx or a 429 — never a permanent 4xx (RM354).

also adds a client's own retried type, such as the rate-limit error gnomad and eutils raise on a 429 before raise_for_status().

Source code in enricher/src/just_dna_enricher/net.py
def retry_if_transient(*also: type[BaseException]) -> retry_base:
    """Retry a transport failure, a timeout, a 5xx or a 429 — never a permanent 4xx (RM354).

    `also` adds a client's own retried type, such as the rate-limit error `gnomad` and `eutils` raise
    on a 429 before `raise_for_status()`.
    """
    return retry_if_exception_type(
        (httpx.TransportError, httpx.TimeoutException, *also)
    ) | retry_if_exception(is_transient_status)

retired_message

retired_message(source: str, response: Response) -> str

What a client's *Retired error says: the URL, the 410, and the source's own text.

For a client that reads the status itself rather than raising on it; raise_if_retired is the same message for one that translates an HTTPStatusError.

Source code in enricher/src/just_dna_enricher/net.py
def retired_message(source: str, response: httpx.Response) -> str:
    """What a client's `*Retired` error says: the URL, the 410, and the source's own text.

    For a client that reads the status itself rather than raising on it; `raise_if_retired` is the
    same message for one that translates an `HTTPStatusError`.
    """
    said = " ".join(_body_text(response).split())[:RETIRED_QUOTE_CHARS] or "(no body)"
    return (
        f"{source} retired the endpoint {response.request.url} (HTTP 410): {said} — this enricher "
        f"reads an API the source no longer serves; upgrade just-dna-enricher"
    )

raise_if_retired

raise_if_retired(
    exc: BaseException, error: type[Exception], source: str
) -> None

Raise error — a client's *Retired type — when exc is a 410; otherwise return.

Called first in a client's status-translating except arm, so the arm keeps translating every other status the way it did.

Source code in enricher/src/just_dna_enricher/net.py
def raise_if_retired(exc: BaseException, error: type[Exception], source: str) -> None:
    """Raise `error` — a client's `*Retired` type — when `exc` is a 410; otherwise return.

    Called first in a client's status-translating `except` arm, so the arm keeps translating every
    other status the way it did.
    """
    if isinstance(exc, httpx.HTTPStatusError) and exc.response.status_code == GONE:
        raise error(retired_message(source, exc.response)) from exc

could_not_ask_reason

could_not_ask_reason(exc: BaseException) -> str

retired when exc says the source retired the endpoint, else unreachable (RM361).

For a handler that caught a client's could-not-ask type and is about to record why the question went unanswered. A retired endpoint is a fact about the source and is cleared by an upgrade; an outage is cleared by a re-run; writing the second about the first sends the reader to the wrong remedy. The exception's causes are read too, so a client that translates a 410 into its plain could-not-ask type (raise XUnavailable(...) from exc) is still read as retired.

Source code in enricher/src/just_dna_enricher/net.py
def could_not_ask_reason(exc: BaseException) -> str:
    """`retired` when `exc` says the source retired the endpoint, else `unreachable` (RM361).

    For a handler that caught a client's could-not-ask type and is about to record why the question
    went unanswered. A retired endpoint is a fact about the source and is cleared by an upgrade; an
    outage is cleared by a re-run; writing the second about the first sends the reader to the wrong
    remedy. The exception's causes are read too, so a client that translates a 410 into its plain
    could-not-ask type (`raise XUnavailable(...) from exc`) is still read as retired.
    """
    seen: BaseException | None = exc
    while seen is not None:
        if isinstance(seen, EndpointRetired) or (
            isinstance(seen, httpx.HTTPStatusError) and seen.response.status_code == GONE
        ):
            return RETIRED
        seen = seen.__cause__
    return UNREACHABLE

aggregate_reason

aggregate_reason(reasons: Iterable[str | None]) -> str

The reason a group of unanswered questions is recorded under: retired if any one was (RM361).

For a record that aggregates several requests: one retired endpoint means the run needs an upgrade whatever else happened, which is the remedy the record must lead with. Otherwise unreachable.

Source code in enricher/src/just_dna_enricher/net.py
def aggregate_reason(reasons: Iterable[str | None]) -> str:
    """The reason a group of unanswered questions is recorded under: `retired` if any one was (RM361).

    For a record that aggregates several requests: one retired endpoint means the run needs an upgrade
    whatever else happened, which is the remedy the record must lead with. Otherwise `unreachable`.
    """
    return RETIRED if RETIRED in set(reasons) else UNREACHABLE

retired_mark

retired_mark(withholder: object) -> int | None

How many 410s withholder has met so far, read before a run asks (RM446).

None for a client that does not count them (a host's own, written before the counter), which withheld_reason then reads as RM361 did: retired for the client's life.

Source code in enricher/src/just_dna_enricher/net.py
def retired_mark(withholder: object) -> int | None:
    """How many 410s `withholder` has met so far, read before a run asks (RM446).

    `None` for a client that does not count them (a host's own, written before the counter), which
    `withheld_reason` then reads as RM361 did: `retired` for the client's life.
    """
    met = getattr(withholder, "retired_met", None)
    return met if isinstance(met, int) else None

withheld_reason

withheld_reason(
    withholder: object, since: int | None = None
) -> str

retired when a withholding client met a 410, else unreachable (RM361).

With since, the mark retired_mark read before this run asked, only a 410 met after it counts (RM446): a reused client's earlier 410 says nothing about why this run went unanswered.

Source code in enricher/src/just_dna_enricher/net.py
def withheld_reason(withholder: object, since: int | None = None) -> str:
    """`retired` when a withholding client met a 410, else `unreachable` (RM361).

    With `since`, the mark `retired_mark` read before this run asked, only a 410 met after it counts
    (RM446): a reused client's earlier 410 says nothing about why this run went unanswered.
    """
    met = retired_mark(withholder)
    if since is not None and met is not None:
        return RETIRED if met > since else UNREACHABLE
    return RETIRED if getattr(withholder, "retired", None) else UNREACHABLE

release_dataset

release_dataset(lane: str, released: str | None) -> str

A dataset label, <lane>_<release>, or the bare lane when the source stated no release (RM361).

The label is a merge-key member and a sources.csv cell; a release the file did not state is withheld from it, never spelled unknown (a release of that name) or None.

Source code in enricher/src/just_dna_enricher/net.py
def release_dataset(lane: str, released: str | None) -> str:
    """A `dataset` label, `<lane>_<release>`, or the bare lane when the source stated no release (RM361).

    The label is a merge-key member and a `sources.csv` cell; a release the file did not state is
    withheld from it, never spelled `unknown` (a release of that name) or `None`.
    """
    return f"{lane}_{released}" if released else lane

stated_release

stated_release(
    dataset: str | None, *lanes: str
) -> str | None

The release a dataset label states, or None when it states none (RM373).

The inverse of release_dataset for a reader: a blank cell, and the bare lane that label writes when the source stated no release, are both no release. What a licence row's dataset is computed from, so a table whose rows say gwas_catalog names no release rather than one called gwas_catalog.

Source code in enricher/src/just_dna_enricher/net.py
def stated_release(dataset: str | None, *lanes: str) -> str | None:
    """The release a `dataset` label states, or `None` when it states none (RM373).

    The inverse of `release_dataset` for a reader: a blank cell, and the bare lane that label writes when
    the source stated no release, are both no release. What a licence row's `dataset` is computed from,
    so a table whose rows say `gwas_catalog` names no release rather than one called `gwas_catalog`.
    """
    label = (dataset or "").strip()
    return None if not label or label in lanes else label

stream_to_file

stream_to_file(
    dest: Path,
    url: str,
    *,
    error_cls: type[Exception],
    retired: type[Exception],
    what: str,
    timeout: float | None = _BULK_TIMEOUT,
    remedy: str = "",
    _stop: Event | None = None,
) -> StreamedFile

Stream url to dest atomically, hashing as it goes; retry the transport, translate the rest.

The one body every download_* in this package calls. Four properties, and each is a defect that reached a user before it was one:

Atomic. The bytes go to <dest>.part and are renamed only once the stream finished, so a failed fetch leaves the directory as it found it rather than truncating a good file already there (@a-failed-fetch-is-not-a-no-op). The partial is removed on failure — a .part left behind is the one residue a re-run would have to reason about.

Retried, but only where retrying is honest. httpx.TransportError covers a connection cut mid-body — RemoteProtocolError subclasses it, which is the failure that motivated this — and a second attempt genuinely fixes it. A status error is not retried: a 404 from a mistyped release tag is the same 404 four times over, and the caller wants it now rather than after three backoffs. attempt_floor reads $JUST_DNA_HTTP_RETRY_ATTEMPTS at retry time like every other policy here, so a deployment can raise the floor without touching code (RM42).

Translated. httpx's exceptions do not leave this function. A caller of this package may not be made to depend on the transport library's exception tree to know that a fetch failed, and the lane adapters catch their own builder's type — so a leak is not merely untidy, it is a lane that cannot report built=False (@client-exception-contract).

Restarted from zero on each attempt. The hasher and the output file are created inside the attempt, not outside it: a retry that appended to a partial body would produce a file whose digest is real and whose contents are nonsense, which no footer check and no raise_for_status would catch.

A 410 is the lane's retired, never its error_cls (RM361, audit round 2). A source that retired the file moved it, and the remedy is an upgrade rather than a re-run; retired is the lane's *Retired (a subclass of error_cls marked EndpointRetired), required so no lane can forget it, and the message is retired_message's (the host that answered, the source's own body quoted), followed by what and the lane's remedy.

Stoppable between chunks, for a caller on a thread pool (RM409). _stop is checked before the request opens and before each chunk is written, and a retry's backoff waits on it rather than sleeping; once it is set the download raises, the .part is removed, and nothing lands under dest. A KeyboardInterrupt reaches only the main thread, so a pool that fetches parts in workers sets the flag on interrupt before it joins them; without it the join waited for every in-flight part to stream to its end, and a 22 GB mirror ignored two Ctrl-C.

Bounded, by default (RM421). timeout is 120 s unless the caller names its own, and httpx applies it to connect, read, write and pool alike. The read half bounds the gap between two chunks, not the whole download, so a slow mirror that keeps sending is never cut off, while a server that accepts the connection and then sends nothing raises httpx.ReadTimeout, which is retried as a transport error above and translated to error_cls on giving up. The default was once None, which httpx reads as no timeout at all: eleven of fourteen callers named none, so such a stall blocked forever, no retry fired, and a pool worker stuck in the read never reached the _stop check. An explicit timeout=None still means no timeout; no caller in this package passes it, and a walk over the call sites keeps it that way.

what names the thing being fetched for the message ("the ClinVar VCF"); remedy is an optional sentence saying what the caller can do instead, for the lanes that accept a local file.

Source code in enricher/src/just_dna_enricher/net.py
def stream_to_file(
    dest: "Path",
    url: str,
    *,
    error_cls: type[Exception],
    retired: type[Exception],
    what: str,
    timeout: float | None = _BULK_TIMEOUT,
    remedy: str = "",
    _stop: threading.Event | None = None,
) -> StreamedFile:
    """Stream `url` to `dest` atomically, hashing as it goes; retry the transport, translate the rest.

    The one body every `download_*` in this package calls. Four properties, and each is a defect that
    reached a user before it was one:

    **Atomic.** The bytes go to `<dest>.part` and are renamed only once the stream finished, so a
    failed fetch leaves the directory as it found it rather than truncating a good file already there
    (`@a-failed-fetch-is-not-a-no-op`). The partial is removed on failure — a `.part` left behind is
    the one residue a re-run would have to reason about.

    **Retried, but only where retrying is honest.** `httpx.TransportError` covers a connection cut
    mid-body — `RemoteProtocolError` subclasses it, which is the failure that motivated this — and a
    second attempt genuinely fixes it. A **status** error is not retried: a 404 from a mistyped
    release tag is the same 404 four times over, and the caller wants it now rather than after three
    backoffs. `attempt_floor` reads `$JUST_DNA_HTTP_RETRY_ATTEMPTS` at retry time like every other
    policy here, so a deployment can raise the floor without touching code (RM42).

    **Translated.** `httpx`'s exceptions do not leave this function. A caller of this package may not
    be made to depend on the transport library's exception tree to know that a fetch failed, and the
    lane adapters catch their own builder's type — so a leak is not merely untidy, it is a lane that
    cannot report `built=False` (`@client-exception-contract`).

    **Restarted from zero on each attempt.** The hasher and the output file are created inside the
    attempt, not outside it: a retry that appended to a partial body would produce a file whose
    digest is real and whose contents are nonsense, which no footer check and no `raise_for_status`
    would catch.

    **A 410 is the lane's `retired`, never its `error_cls`** (RM361, audit round 2). A source that
    retired the file moved it, and the remedy is an upgrade rather than a re-run; `retired` is the
    lane's `*Retired` (a subclass of `error_cls` marked `EndpointRetired`), required so no lane can
    forget it, and the message is `retired_message`'s (the host that answered, the source's own body
    quoted), followed by `what` and the lane's `remedy`.

    **Stoppable between chunks, for a caller on a thread pool** (RM409). `_stop` is checked before the
    request opens and before each chunk is written, and a retry's backoff waits on it rather than
    sleeping; once it is set the download raises, the `.part` is removed, and nothing lands under `dest`. A `KeyboardInterrupt` reaches only the main thread, so a
    pool that fetches parts in workers sets the flag on interrupt before it joins them; without it the
    join waited for every in-flight part to stream to its end, and a 22 GB mirror ignored two Ctrl-C.

    **Bounded, by default** (RM421). `timeout` is 120 s unless the caller names its own, and httpx
    applies it to connect, read, write and pool alike. The read half bounds the gap between two chunks,
    not the whole download, so a slow mirror that keeps sending is never cut off, while a server that
    accepts the connection and then sends nothing raises `httpx.ReadTimeout`, which is retried as a
    transport error above and translated to `error_cls` on giving up. The default was once `None`, which
    httpx reads as no timeout at all: eleven of fourteen callers named none, so such a stall blocked
    forever, no retry fired, and a pool worker stuck in the read never reached the `_stop` check. An
    explicit `timeout=None` still means no timeout; no caller in this package passes it, and a walk over
    the call sites keeps it that way.

    `what` names the thing being fetched for the message ("the ClinVar VCF"); `remedy` is an optional
    sentence saying what the caller can do instead, for the lanes that accept a local file.
    """
    dest = Path(dest)
    dest.parent.mkdir(parents=True, exist_ok=True)
    tmp = dest.with_name(dest.name + ".part")

    def _stopped() -> None:
        if _stop is not None and _stop.is_set():
            raise _StreamStopped(f"stopped downloading {what}")

    # A stoppable download's backoff waits on the flag instead of sleeping, so a stop that lands
    # between attempts is not held for up to thirty seconds of backoff before the next one notices it.
    nap = {"sleep": _stop.wait} if _stop is not None else {}

    @retry(
        before_sleep=_log_retry,
        retry_error_callback=_log_give_up,
        stop=attempt_floor(3),
        wait=_jittered_backoff(base=2.0, cap=30.0),
        retry=retry_if_exception_type(httpx.TransportError),
        reraise=True,
        **nap,
    )
    def _attempt() -> StreamedFile:
        hasher = hashlib.sha256()
        _stopped()
        with httpx.stream("GET", url, follow_redirects=True, timeout=timeout) as response:
            if response.status_code == GONE:
                response.read()  # a 410's body is the source's own word on where the file went
            response.raise_for_status()
            etag = response.headers.get("ETag")
            last_modified = response.headers.get("Last-Modified")
            with tmp.open("wb") as handle:
                for chunk in response.iter_bytes():
                    _stopped()
                    handle.write(chunk)
                    hasher.update(chunk)
        return StreamedFile(tmp, hasher.hexdigest(), etag, last_modified)

    logger.info("Downloading %s from %s ...", what, url)
    try:
        streamed = _attempt()
    except httpx.HTTPError as exc:
        tmp.unlink(missing_ok=True)
        if isinstance(exc, httpx.HTTPStatusError) and exc.response.status_code == GONE:
            # The host that answered retired it (a redirect may have moved the request), and the lane's
            # own remedy still applies: a release the operator picked, or a file they already hold.
            gone = retired_message(exc.response.request.url.host, exc.response)
            raise retired(
                f"{gone} (fetching {what}). {remedy}" if remedy else f"{gone} (fetching {what})"
            ) from exc
        message = f"could not download {what} from {url}: {exc}"
        raise error_cls(f"{message}. {remedy}" if remedy else message) from exc
    except BaseException:
        # Not only the transport: a disk that fills or a mount that turns read-only raises `OSError`
        # from `handle.write`, and the partial was left behind on exactly that leg while the
        # docstring promised none. The type is the caller's (the lane adapters catch `OSError`),
        # so it is not translated — only the residue is removed.
        tmp.unlink(missing_ok=True)
        raise
    tmp.replace(dest)
    logger.info("Downloaded %s (sha256 %s)", dest, streamed.sha256)
    return dataclasses.replace(streamed, path=dest)

retry_after_seconds

retry_after_seconds(
    exc: BaseException | None,
    *,
    now: Callable[[], float] = time.time,
) -> float | None

How long a throttled answer asked the client to wait, or None when it did not say.

Read only off a status a retry can change (is_transient_status: a 5xx or a 429), in either of RFC 9110's forms — delta-seconds (Ensembl sends a fraction, 40.0) or an HTTP-date, measured against now (wall clock, since the date is one). A date already past is 0.0. No header, an unreadable one, or any other exception is None: unknown, never a wait of zero.

Source code in enricher/src/just_dna_enricher/net.py
def retry_after_seconds(exc: BaseException | None, *, now: Callable[[], float] = time.time) -> float | None:
    """How long a throttled answer asked the client to wait, or `None` when it did not say.

    Read only off a status a retry can change (`is_transient_status`: a 5xx or a 429), in either of
    RFC 9110's forms — delta-seconds (Ensembl sends a fraction, `40.0`) or an HTTP-date, measured
    against `now` (wall clock, since the date is one). A date already past is `0.0`. No header, an
    unreadable one, or any other exception is `None`: unknown, never a wait of zero.
    """
    if not isinstance(exc, httpx.HTTPStatusError) or not is_transient_status(exc):
        return None
    raw = (exc.response.headers.get("Retry-After") or "").strip()
    if not raw:
        return None
    try:
        seconds = float(raw)
    except ValueError:
        try:
            when = email.utils.parsedate_to_datetime(raw)
        except (TypeError, ValueError):
            return None
        if when.tzinfo is None:
            # `-0000` parses naive; an HTTP-date is GMT by definition, never the host's local time.
            when = when.replace(tzinfo=datetime.UTC)
        seconds = when.timestamp() - now()
    if not math.isfinite(seconds):
        return None
    return max(seconds, 0.0)

asks_past_ceiling

asks_past_ceiling(
    exc: BaseException | None,
    ceiling: float = RETRY_AFTER_CEILING,
) -> bool

Whether exc asked for a wait longer than ceiling — a request not to retry now.

Source code in enricher/src/just_dna_enricher/net.py
def asks_past_ceiling(exc: BaseException | None, ceiling: float = RETRY_AFTER_CEILING) -> bool:
    """Whether `exc` asked for a wait longer than `ceiling` — a request not to retry now."""
    asked = retry_after_seconds(exc)
    return asked is not None and asked > ceiling

retry_if_transient_within

retry_if_transient_within(
    *also: type[BaseException],
    ceiling: float = RETRY_AFTER_CEILING,
) -> retry_base

retry_if_transient, except a throttle whose Retry-After exceeds ceiling is not retried.

Source code in enricher/src/just_dna_enricher/net.py
def retry_if_transient_within(*also: type[BaseException], ceiling: float = RETRY_AFTER_CEILING) -> retry_base:
    """`retry_if_transient`, except a throttle whose `Retry-After` exceeds `ceiling` is not retried."""
    return retry_if_transient(*also) & retry_if_exception(lambda exc: not asks_past_ceiling(exc, ceiling))