just_dna_enricher.placement¶
just_dna_enricher.placement ¶
Two authorities' placements of one rsID, compared before a row is written (RM267).
The gap this closes. The resolver chain asks its links in order and keeps the first answer: the
Ensembl snapshot, then the ClinVar snapshot for what Ensembl missed, then live Ensembl and gnomAD. The
same run already holds the ClinVar snapshot (for the clin_sig cross-check) and may hold ClinVar-GKM
(RM305), yet nothing compared a placement one link gave against another's. So every coordinate in
resolution.csv had one witness, and authority=ensembl read the same whether a second authority
agreed, disagreed or was never asked. S117's case is the one that matters: Ensembl's VCF dump anchors
rs8176719 (ABO O1) at 9:133257520 G>GC, a different haplotype from ClinVar's 9:133257521 T>TC
(GRCh38 reads GGGG T ACC at 517–524). The wrong anchor base is a real base, so the reference-allele
check passes it, and RM31's tolerance calls the position difference undecided.
The comparison, per allele, in the order that needs least. For each allele the first authority gives, the witness's counterparts are its alleles for the same rsID on the same contig, within the window an indel can slide in, with the same kind of event: an insertion or deletion of the same length, or a substitution to the same bases. A counterpart is the same event when
- the spellings are identical (nothing needed);
- both are substitutions — a substitution has one spelling once its shared flank is trimmed
(
parsimony_reduce's rule), so the trimmed spellings decide it; - both mint a VRS id, compared exactly — the id is over the fully justified allele, so it needs no
tolerance (RM270's probe: a respelling mints one id,
rs8176719's pair mints two). We mint both sides ourselves: ClinVar-GKM's own ids are VRS 2.1.1 and ours 2.0 (ClinVarGkmWitness.ids_comparable), so its VCF spelling is minted here rather than its id compared; - both left-align against the reference window (
sequences._left_align), compared as tuples; - otherwise the comparison is undecided — counted, never read as agreement (
@tautology-zero).
The ladder is events.same_event, the one judge every comparison of two spellings in this tier asks
(RM375): an authored coordinate against its resolved locus, and a re-derived answer against a recorded one.
An allele with no counterpart is unplaced by that witness: the witness holds a different allele of the rsID, or none. That is not a disagreement — ClinVar holds the alleles somebody submitted, not every allele dbSNP knows — and it is the class whose count is published as "one witness".
What a disagreement does. The locus holding the disputed allele is withheld: no row is written for
it, and the finding names both placements. A locus of the same rsID nobody disputes is still written.
When every locus is withheld the caller routes the subject like an unreachable one — an rsID-only
subject stays unresolved, a subject that authored its own coordinate keeps it. Nothing is
rewritten (PROPOSAL_0_8_PT2 §4 question 3: no corrected spelling in 0.8, @enrichment-is-validation).
Independence. A placement the ClinVar link filled has no independent witness here — ClinVar-GKM is ClinVar's own release — so it is counted with the one-witness class and never compared.
dbSNP, the third witness, online only (RM398). Neither loaded witness holds S117's own rs8176719
(ClinVar has no record there, ClinVar-GKM lacks it), so against them it is one witness. dbSNP does: the
esummary db=snp record the merge check already fetches for every rsID carries dbSNP's GRCh38 SPDI,
NC_000009.12:133257521::C, which anchors to 9:133257521 T>TC. So for an indel the Ensembl snapshot
placed and no loaded witness holds, an online run with the merge check on reads that record first
(_dbsnp_placements; the merge check is then served the same records and asks nothing again) and
compares it by the same ladder. The witness is named dbsnp, the source placement_witness joins
sources.csv on. The snapshot was built from dbSNP, so a comparison against it is self-consistent,
not independent (RM292): it catches the snapshot's processing disagreeing with dbSNP's own placement,
which is S117's case, but an agreement proves nothing about dbSNP's placement, so it stays in the
one-witness count (_UPSTREAM_OF). A record that could not be fetched, or an indel whose anchor base
could not be read, leaves dbSNP not asked for that rsID, counted, never agreement
(@unreachable-not-absent); --offline and a run with the merge check off ask nothing.
Until RM398 the third witness was live Ensembl REST (ensembl), asked once per rsID: S133 measured
REST's cold requests as most of an online re-derivation's time, and dbSNP's placement is in a record the
run fetched anyway. ENSEMBL_WITNESS stays in the roster because recorded rows name it; nothing asks it.
live_witness_subjects still chooses the subjects, and ask_live_witness stays as 0.8.0 shipped it.
PlacementDisagreement
dataclass
¶
PlacementDisagreement(
rsid: str,
authority: str,
placed: tuple[str, ...],
witness: str,
witness_placed: tuple[str, ...],
)
An rsID two authorities place as two different events. Both placements are named.
PlacementCheck
dataclass
¶
PlacementCheck(
witnesses: tuple[str, ...] = (),
subjects: int = 0,
disagreements: list[PlacementDisagreement] = list(),
withheld_loci: set[
tuple[str, tuple[str, int, str, str]]
] = set(),
undecided: list[str] = list(),
single_witness: list[str] = list(),
self_consistent: list[str] = list(),
not_asked: list[str] = list(),
not_asked_reason: str | None = None,
reported_by: set[str] = set(),
placed_by: set[str] = set(),
placed: int = 0,
rows: dict[
tuple[str, tuple[str, int, str, str]],
tuple[str, str],
] = dict(),
not_checked: str | None = None,
_loci_of: dict[
str, list[tuple[str, int, str, str]]
] = dict(),
)
What the comparison did (RM267) — the PairCheck shape, so an empty list cannot read as a pass.
witnesses— the second authorities this run had loaded or asked, sorted; empty means none was.subjects— rsIDs compared: at least one allele a witness asked about it also placed. The record's denominator.disagreements— one per (allele, witness) placed as a different event. Its locus is withheld.withheld_loci—(rsid, locus_key)of every locus holding a disputed allele; never written.undecided— rsIDs with no withheld locus left unsettled: some allele could not be decided.single_witness— rsIDs this run placed that no independent witness placed: the ClinVar link's own fills, every rsID none of whose alleles a witness holds, and every rsID only its own authority or its upstream compared (dbSNP against the snapshot). Published beside the finding (@tautology-zero).self_consistent— compared rsIDs whose only witness was their own authority or its upstream (dbSNP for a snapshot placement): a disagreement there is a finding, an agreement proves nothing.not_asked— rsIDs the dbSNP witness was due to be asked about and got no answer for: unknown.not_asked_reason— why, when one reason covers them all (the merge check switched off, or dbSNP unreachable);Nonemeans per rsID: an indel's anchor base not read.reported_by— the authorities whose placements were compared, the record's checked side.placed_by— the authorities of every placement looked at, which a skip names instead.placed— every rsID the comparison looked at, the denominatorsingle_witnessis a part of.rows—(rsid, locus_key) -> (witness, verdict)for the row writer.not_checked— aVALID_VERIFICATION_SKIPSkey when the comparison could not run at all.
contested
property
¶
The rsIDs with at least one disputed placement, sorted — the record's findings.
withheld
property
¶
The rsIDs none of whose placements may be written, sorted: every locus held a disputed allele.
keeps ¶
Whether a locus the first authority gave may be written (no allele of it is disputed).
row_columns ¶
(placement_witness, placement_verdict) for one locus about to be written.
Source code in enricher/src/just_dna_enricher/placement.py
sentences ¶
The counts a reader needs beside the finding, one sentence per class, never per row.
Source code in enricher/src/just_dna_enricher/placement.py
locus_key ¶
How a row's verdict is found again after the hosting filter: the locus as the link gave it.
Source code in enricher/src/just_dna_enricher/placement.py
compare_placements ¶
compare_placements(
placed: Mapping[str, Sequence[Mapping[str, object]]],
authority_of: Mapping[str, str | None],
witnesses: Mapping[
str, Mapping[str, Sequence[Mapping[str, object]]]
],
*,
mint: Mint,
read_window: ReadWindow | None,
asked_about: Mapping[str, Collection[str]]
| None = None,
) -> PlacementCheck
Compare every rsID in placed against every loaded witness it is independent of.
placed is rsid -> loci as the first authority gave them (before the hosting filter: the
question is where the source puts the rsID, not which locus the module can use); authority_of
names that authority per rsID; witnesses is witness name -> rsid -> loci. A witness in
asked_about was asked only about the rsIDs it lists there (the dbSNP witness); every other
rsID's row does not name it. A loaded snapshot witness is asked about every rsID.
A witness that is the rsID's own authority, or its upstream, compares, but witnesses nothing
independently (dbSNP for a snapshot placement, _UPSTREAM_OF): its disagreement withholds like any
other, and its agreement leaves the rsID in the one-witness count.
Withheld per locus, judged per allele. A locus holding a disputed allele is not written; a
locus of the same rsID nobody disputes still is, so one contested allele does not cost the module
a placement both authorities agree on. A written row reads same_event only when every one of
its ALTs was placed the same by a witness — an ALT no witness holds makes the row unplaced, an
unsettled one undecided, so no allele reads as confirmed that was not.
Source code in enricher/src/just_dna_enricher/placement.py
281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 | |
load_witnesses ¶
load_witnesses(
clinvar_ref: Path | None,
clinvar_gkm_ref: Path | None,
rsids: Sequence[str],
) -> dict[str, Loci]
witness name -> rsid -> loci from every second authority located for this run, read locally.
A located snapshot that will not answer is a witness this run does not have, logged and left out — the same footing the ClinVar link gives it. It is never an empty answer: an unusable witness that read as "holds nothing" would put every rsID in the one-witness count for the wrong reason.
Source code in enricher/src/just_dna_enricher/placement.py
live_witness_subjects ¶
live_witness_subjects(
placed: Mapping[str, Sequence[Mapping[str, object]]],
link_of: Mapping[str, str | None],
loaded: Mapping[
str, Mapping[str, Sequence[Mapping[str, object]]]
],
) -> list[str]
The rsIDs the third witness is asked about, sorted: placed by the Ensembl snapshot, with at least one allele that is not a substitution, and held by no loaded witness. dbSNP since RM398, Ensembl REST before it; the rule did not move.
The snapshot only, because S117's class lives in its dump, and a row the live rung gave is REST's own answer. Indels only, because a substitution has one spelling and the snapshot cannot respell it. And only where no loaded witness holds the rsID, because there a self-consistent answer adds nothing beside an independent one. Widening it to every placed rsID is RM406.
Source code in enricher/src/just_dna_enricher/placement.py
ask_live_witness ¶
Ask the live REST rung (EnsemblResolver.resolve_rsid_rest) about each rsID, one at a time.
As 0.8.0 shipped it. The run no longer calls it: the third witness is dbSNP since RM398
(_dbsnp_witness).
Returns rsid -> loci for every rsID it answered (an empty list is an answer: REST has no placeable
locus), and the rsIDs it did not answer — None loci, whether the request failed after its retries
or every mapping's anchor base was unreadable (RM268). Those are not asked, never asked-and-empty.
Nothing is journalled: a witness's answer is not a placement this run writes.
Source code in enricher/src/just_dna_enricher/placement.py
answered_placements ¶
The resolution.csv subjects whose overlay states a whole placement (RM267), read-only.
Per field, because it silences (@a-set-that-silences-is-narrower-than-one-that-raises): a
disputed rsID has no row to update, so the author's answer is an insert, one overlay row per
field, and it counts only when insert and update rows for one (subject, member) cover every
PLACEMENT_FIELDS cell. A malformed overlay answers nothing; the compile raises on it.