just_dna_enricher.clinvar_gkm¶
just_dna_enricher.clinvar_gkm ¶
ClinGen's ClinVar-GKM, cut to its indels: an offline witness for where an indel sits (RM305).
What this lane is for, and what it is not. Three checks need to know whether a placement is right
(RM267's two authorities, RM270's respelling, RM292's named witness), and each compares today against a
source that shares our failure. ClinVar-GKM republishes ClinVar in GA4GH form, through a pipeline that
is not ours, with a spelling-independent VRS id on every allele. That makes it a witness: the lane
answers where does an independent party place this indel, and nothing else. It runs no check of its
own; a check names it as what it checked against. Clinical significance still comes from the clinvar
lane.
Indels only (decided 2026-11-04). The ~4.6 M alleles are mostly substitutions, whose placement is
not in doubt and whose ids the format tier mints with the stdlib. What is kept is every allele whose
SPDI deletes and inserts a different number of bases: a net insertion or a net deletion. That drops
substitutions, same-length MNVs, and the reference-agreeing ReferenceLengthExpression alleles
(NC_000021.9:g.45504512_45504520=, deleting and inserting the same sequence), whichever state type
carries them. An allele with no SPDI expression (GRCh37- or transcript-only names) and one off the
GRCh38 primary assembly (one T2T accession) are dropped too; the build counts each class.
Where the bytes come from. One Parquet set under https://data.clinvar-gkm.org/v1/1.1/parquet/,
overwritten in place every week, with no dated or archived copies (docs/probes/LANES_0_8.md §3.1).
So a release cannot be pinned by URL. The pin is (source_release, sha256) from the set's own
manifest.json, and the builder refuses bytes that do not match it: a file overwritten between the
manifest read and the download is drift, and a build from it would describe one release with
another's bytes. Both inputs are downloaded whole and read locally, which is what the source asks
("Download the Parquet files you need, then query them locally. The R2 hosting has rate limits and is
designed for file downloads, not as a remote query endpoint"). A whole download also keeps the one
failure the probe saw off this path: an R2 range request that came back 200 with the whole file. A
truncated or partial body cannot pass the size and digest check either way.
Two keys every row has, and one most rows have. The VRS id and the gnomAD-style VCF spelling
(4-87310240-T-TA) are on every allele; the rsID is on about 71% of the indels (LANES §3.5), and lives
in variation.parquet, joined through the variation's DefiningAlleleConstraint. A reader keys on the
first two and treats the rsID as secondary.
The VRS version is declared, not measured. ClinVar-GKM's 1.0.0 changelog says its ids are minted
under VRS 2.1.1; neither the Parquet nor its manifest carries a version (LANES §3.6). So release.json
records vrs_version with the basis beside it. This tier mints under 2.0 (RM304), and 2.0 and 2.1 move
the ids of some ambiguous insertions, so a VRS-id comparison against this witness is only sound when
the minor matches (ids_comparable). Until vrs-python ships 2.1, the VCF spelling is the key a
comparison uses on every row; the caller supplies it left-normalized, since the gnomad expression is.
ClinVarGkmError ¶
Bases: RuntimeError
The ClinVar-GKM lane could not be built or read.
ClinVarGkmUnavailable ¶
Bases: ClinVarGkmError
The source could not be asked at all: a failed request, never "it published nothing".
A subclass so an except ClinVarGkmError keeps catching it, and a caller that has to tell
unreachable from unreadable can (@client-exception-contract): the narrow arm goes first.
ClinVarGkmRetired ¶
Bases: ClinVarGkmUnavailable, EndpointRetired
The source answered HTTP 410: it retired the file asked for (RM354).
A fact about the source, not an outage, and not retried. A ClinVarGkmUnavailable so every existing
handler still catches it; the message says the file moved rather than that the host was down.
ClinVarGkmMismatch ¶
Bases: ClinVarGkmError
The bytes are not the ones the manifest states: the set was overwritten, or the body is partial.
ManifestFile
dataclass
¶
One file of the published Parquet set, as manifest.json states it.
GkmManifest
dataclass
¶
GkmManifest(
source_release: str,
files: dict[str, ManifestFile],
dataset: str | None = None,
build_date: str | None = None,
schema_version: str | None = None,
license_id: str | None = None,
license_scope: str | None = None,
attribution: str | None = None,
disclaimer: str | None = None,
data_use_policy: str | None = None,
)
parquet/manifest.json: which release the one Parquet set holds, and each file's digest.
source_release is the ClinVar release the set was built from, and the half of the pin a label
is made of. The licence fields are kept verbatim because the snapshot's LICENSE.txt quotes them.
file ¶
The manifest's entry for name, or a refusal: a build cannot verify a file it does not list.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
ClinVarGkmClient
dataclass
¶
ClinVarGkmClient(
manifest_url: str = PARQUET_MANIFEST_URL,
index_url: str = RELEASE_INDEX_URL,
timeout: float = 30.0,
client: Client | None = None,
gate: PacingGate | None = None,
)
The two small JSON documents that say what the source publishes now.
parquet_manifest() is the Parquet set's own record; latest_release() is the release line's
index, whose latest_release is the newest weekly release. They differ: the Parquet set is
rebuilt from a weekly release a few days after it lands, so on 2026-11-04 the set held 2026-10-24
while the line's newest release was 2026-10-31 (LANES §3.2). @client-exception-contract: transport
and status failures are retried, then arrive as ClinVarGkmUnavailable, never as an httpx type.
file_url ¶
A file of the Parquet set: it sits beside the manifest that describes it.
parquet_manifest ¶
The published Parquet set's manifest. Raises ClinVarGkmUnavailable when unreachable.
latest_release_or_none ¶
latest_release, withheld as None when the index cannot be read.
The index answers one informational question (does the Parquet set lag the line?). Its
outage may not speak for the manifest's questions (@one-registrys-outage-may-not-speak-for-
another), so a build and a currency check go on without it and say the lag is unknown.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
latest_release ¶
The release line's newest weekly release, or None when the index states none.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
current_release ¶
The currency probe: the label a rebuild today would write, read off the live manifest.
The Parquet set's release, not the line's newest weekly one. A module drafted against this
lane can only ever be as current as the set, so comparing it against a release no Parquet
carries would report every module behind for the week the set lags, with nothing to re-draft
from. The set's own lag is LaneCurrency.source_lags, kept apart.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
LaneCurrency
dataclass
¶
LaneCurrency(
pinned_release: str | None,
pinned_sha256: tuple[tuple[str, str | None], ...]
| None,
published_release: str | None = None,
published_sha256: tuple[tuple[str, str | None], ...]
| None = None,
latest_release: str | None = None,
unchecked: str | None = None,
)
A built snapshot against what the source publishes now — three questions, kept apart.
behind: the source has published a newer Parquet set, so a rebuild would get newer bytes.drifted: the source's set names the same release with different bytes. The pin caught an in-place overwrite the label cannot see.source_lags: the published set is older than the line's newest weekly release. Nothing here can fix that (no Parquet exists for the newer release), so it is reported, never folded intobehind: a rebuild would change nothing.
Each is tri-state; unchecked names why none could be answered (@currency-asks-the-source-not-
the-cache: the published release comes over the wire, never from the snapshot's own record).
sentences ¶
What an operator is told: each answered question that found something, and every unknown.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
WitnessPlacement
dataclass
¶
WitnessPlacement(
vrs_id: str,
chrom: str,
pos: int,
ref: str,
alt: str,
gnomad: str,
spdi: str,
state_type: str | None,
clinvar_variation_id: str | None,
rsid: str | None,
)
Where ClinVar-GKM places one allele: one row of the slice.
Flat, one row per (allele, rsID): an allele with two rsIDs is two rows (4 alleles on
2026-10-24), and an rsID naming two alleles (a multi-allelic site) returns both. A lookup never
picks one (@multiplicity-is-a-finding).
vcf_key
property
¶
chrom:pos:ref:alt, the coordinate form variant_key uses for one ALT.
ClinVarGkmWitness
dataclass
¶
The loaded slice, and the three lookups a check makes against it.
[] from any lookup means not witnessed, never placed wrong: the lane holds indels only, a
ClinVar-GKM release holds only what ClinVar holds, and 29% of the indels carry no rsID.
vrs_version
property
¶
The VRS version the witness's ids were minted under, as release.json declares it.
ids_comparable ¶
Whether a VRS id we minted under our_vrs_spec may be compared to the witness's ids.
Tri-state at the minor, the grain the id rule changes at (RM304): None when either side is
unstated. When it is not True, compare by the normalized VCF spelling instead.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
by_rsids ¶
by_rsid over many rsIDs in one filter, keyed by the rsID as asked; an absent one is absent.
One pass over the slice rather than one per rsID: a module asks about every rsID it places (RM267), and the slice is every ClinVar indel.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
by_vcf ¶
Exact match on the left-normalized VCF spelling. The caller normalizes; this never does.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
lookup ¶
A variant_key, a VRS id or an rsID → the placements the witness holds for it.
A coordinate key must carry exactly one ALT (chrom:start:ref:alt): a position-only key names
no allele, and a multi-ALT key names several, so both are refused rather than half-answered.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
parquet_url ¶
parse_manifest ¶
Read manifest.json. A missing release or file digest is a refusal, never a default.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
clinvar_gkm_release_label ¶
clinvar_gkm_<source_release>, or None when no release is stated.
One formula, called by the builder that writes dataset and by the currency probe that reads the
live manifest, so a snapshot and the source name one release with one string.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
release_older ¶
Tri-state first < second over two YYYY-MM-DD dates; None when either is not one.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
bare_digest ¶
A digest without its sha256: prefix(es), lower case; None for blank.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
lane_currency ¶
lane_currency(
snapshot_dir: Path,
*,
client: ClinVarGkmClient | None = None,
offline: bool = False,
) -> LaneCurrency
Ask the source whether snapshot_dir is current. Reads the snapshot; writes nothing.
Source code in enricher/src/just_dna_enricher/clinvar_gkm.py
normalize_chrom ¶
A contig as the gnomad expression spells it: no chr, upper case, M read as MT.