just_dna_enricher.sequences¶
just_dna_enricher.sequences ¶
Reference-sequence access, and the reference-allele check it makes possible.
This is the one place in the workspace that reads actual bases. Two callers need it: VRS indel
normalization (vrs.py), which must justify an allele against its surroundings, and the
reference-allele check below, which is the reason this module is more than plumbing.
Enrichment is partly validation. The enricher is the only tier that can compare authored data
against reality — the format and compiler tiers are inject-only by charter (Principle 2) and have no
reference to check anything against. So surfacing a discrepancy between what a module claims and what
the genome says is a goal of this tier, not a side effect of it. verify_reference_alleles is the
first check of that kind; alts-level and cross-source checks are the natural next ones.
It reports; it never repairs. A mismatch is surfaced with both values and left in place. Silently
rewriting an authored ref would destroy the evidence that something upstream is wrong — a liftover
run against the wrong assembly, an off-by-one, a hand-edited row — and would turn a loud data problem
into a quiet one. The author (or their pipeline) decides what the correct value is; the enricher's job
is to make sure they know there is a decision to make.
Why this check has to exist at all. A GA4GH VRS allele id is built from which sequence, which
interval, and what replaces it — the reference allele is not one of its components, because the
refget accession plus the interval already determine it (sequence[start:end] has exactly one answer).
That is correct and deliberate: a content-addressed identity must be a function of the allele, not of
the claim about it. But it means an authored ref is unchecked by minting. Two consequences, both
real:
- a wrong ref base is absorbed silently —
11:5227002 C>Aand the trueT>Amint the same id; - a wrong ref length is worse — it changes the interval, so it mints a well-formed id for a different allele, with nothing downstream able to notice.
VCF gets this check for free, because its CHROM is a name rather than a digest and REF is therefore
load-bearing. VRS trades that redundancy away for canonicality. This module buys the check back on the
one tier that has the sequence to do it with.
SequenceProxy
dataclass
¶
SequenceProxy(
uri: str = DEFAULT_SEQREPO_URI,
offline: bool = False,
_proxy: object = None,
_tried: bool = False,
_cache: dict[tuple[str, int, int], str | None] = dict(),
)
Lazy, cached access to reference bases. offline makes every read return None.
Two properties earn the wrapper over create_dataproxy directly: the proxy is built at most
once and only if actually needed (a module of pure substitutions never touches the network for
minting), and reads are memoized by (accession, start, end) — a module commonly asks about
the same locus several times, and each miss is an HTTP round trip.
proxy ¶
The underlying ga4gh.vrs data proxy, or None when offline or unreachable.
None is a normal outcome, not an error: an offline run legitimately has no sequence access,
and callers degrade (leave an id unminted, skip a check) rather than fail.
Source code in enricher/src/just_dna_enricher/sequences.py
subsequence ¶
Uppercased reference bases over the interbase interval [start, end), or None.
Source code in enricher/src/just_dna_enricher/sequences.py
SpellingVerdict
dataclass
¶
SpellingVerdict(
verdict: bool | None,
normalized: tuple[
tuple[str, int, str, str] | None, ...
],
)
Whether a recorded spelling is the left-aligned one, per ResolutionRow.left_aligned (RM270).
verdict is Kleene over the ALTs: False when any ALT has another leftmost spelling, else None
when any could not be decided, else True. normalized is the leftmost spelling of each ALT the
run decided, in ALT order, None where it could not.
SpellingCheck
dataclass
¶
SpellingCheck(
respelled: list[str] = list(),
findings: list[str] = list(),
checked: int = 0,
unknown: int = 0,
)
What RM270's spelling pass did in one enrich run — never an empty list read as a pass.
respelled—"<rsid> <as the link gave it> → <as written>", one per locus this run filled and wrote in another spelling than its link's. A fill is outsidecontent_signature(@rm43-positional-fill), so writing it left-aligned is legal; it reaches a recorded table only on--rederive(@sidecar-authoritative).findings—"<variant_key>: <authored> → <left-aligned>", one per authored coordinate whose leftmost spelling differs. Reported, never rewritten (@enrichment-is-validation): the cell is the row's identity, and the row'svrs_idis what joins it to a caller's spelling.checked— authored coordinates judged either way, the findings' denominator.unknown— authored coordinates no verdict was reached for (an indel with no reference read, a non-nucleotide allele), counted apart so an offline run never reads as a clean bill.
judge ¶
Count one authored coordinate and, when it is not left-aligned, name its leftmost spelling.
Source code in enricher/src/just_dna_enricher/sequences.py
NormalizedLocus
dataclass
¶
A filled locus in its left-aligned spelling (normalize_locus).
loci— one or more loci, each in the leftmost parsimonious spelling of the ALTs it holds; the locus unchanged whenalignedisNone.aligned—Truewhen every ALT was decided (so every piece is left-aligned),Nonewhen any could not be, in which case nothing was respelled.piece_of— for each ALT of the source locus, in its order, the index intolociit landed in.
RefMismatch
dataclass
¶
RefMismatch(
variant_key: str,
chrom: str,
start: int,
claimed: str,
actual: str,
genome_build: str = "GRCh38",
shift: int | None = None,
)
One row whose authored reference allele disagrees with the reference sequence.
A mismatch has two quite different causes, and the finding names which one it found. Either the
ref cell is wrong at a position that is right, or — the case that actually shows up in the wild
— the position is wrong and ref was correct all along for the variant the author meant. The
second is what a pos - 1 conversion produces, and reporting it as a bad ref sends the author
to the wrong column while leaving them to wonder why their coordinate "validated" against dbSNP.
distorts_the_allele_id
property
¶
Whether the mismatch also makes the minted VRS id describe a different allele.
Three cases, and the middle one is why this is not simply a length test:
- A single-base claim at a position that is right: no. The interval is one base wide whichever base the author thought was there, so the minted id is the true allele at that position — correct, despite the row being wrong. Only this check can surface it.
- A shifted coordinate: yes, whatever the length. The id is minted at the authored position, so a shift means it addresses a base the author never meant. The id is still a correct digest of what it was given, which is exactly why nothing downstream can catch it: the compiler's VRS pass recomputes the same wrong id and reports it verified.
- A longer claim: yes. The claimed length sets the interval, so a wrong
refmeans the allele spans the wrong bases and names an event the author did not intend.
diagnosis
property
¶
The cause, as far as it was established — the grouping key for a run's summary.
RefCheck
dataclass
¶
RefCheck(
mismatches: list[RefMismatch],
subjects: int = 0,
not_checked: str | None = None,
unread: int = 0,
)
What the reference-allele check did, not just what it found (RM45).
The mismatches alone answer one of the two questions a reader has, and the run used to discard the
other: this function returns [] for no row disagreed, for there was no sequence access, and
for every row was a symbolic allele nothing could be compared, and those are three different
statements about a module. subjects is what was actually read and compared, and not_checked
carries a VALID_VERIFICATION_SKIPS key when the whole pass could not run. Same shape, same
argument, as EnrichmentResult.clin_sig_not_checked — extended here because verification.json
publishes it, and publishing an empty finding list with no denominator is what RM45 exists to stop.
left_normalize ¶
left_normalize(
chrom: str,
pos: int,
ref: str,
alt: str,
read_window: Callable[[str, int, int], str | None]
| None,
) -> tuple[str, int, str, str] | None
One ALT → its leftmost parsimonious VCF spelling, or None when this run cannot decide it (RM270).
The spelling a caller writes after bcftools norm -m -any, so an exact (chrom, pos, ref, alt)
join meets it. A substitution or MNV has one spelling once its shared flank is trimmed, so it needs
no reference read and is decided offline. An insertion or deletion can slide inside a repeat, so it
goes through _left_align, the one normalizer RM267's comparison also uses
(@one-normalizer-two-spellings); with no read_window, or an unreadable one, it is None. A
non-nucleotide allele (symbolic, ., *, an N) names no spelling to align, and an ALT equal to
its REF names no event: both are None, never a guess.
Source code in enricher/src/just_dna_enricher/sequences.py
per_alt_loci ¶
Alleles (chrom, start, ref, alt) → loci, each ALT in its own spelling (trim_allele) (RM381).
The one way a reader turns a source's rows into the locus dicts the chain passes on. ALTs whose
trimmed (chrom, start, ref) agree share a locus, their alts comma-joined, sorted and deduplicated;
ALTs that trim to different refs are different loci, the split a caller's bcftools norm -m -any
makes. Loci come back sorted by (chrom, start, ref) (P7). So the split repo's
16:23635706 GTT>G / GTT>GT / GTT>GTTT is three loci, G>GT, GT>G and GTT>G, and never the
padded GTT>G,GT,GTTT, in which a genotype would be read in a frame no caller writes.
Source code in enricher/src/just_dna_enricher/sequences.py
spelling_verdict ¶
spelling_verdict(
chrom: str | None,
start: int | None,
ref: str | None,
alts: str | None,
read_window: Callable[[str, int, int], str | None]
| None,
) -> SpellingVerdict
Check a recorded locus against its left-aligned spelling, ALT by ALT, and never rewrite it.
A multi-allelic cell is judged per ALT, as a caller splitting it would read it: CAG>C,CAGAG is
the jointly trimmed spelling of a deletion and an insertion, and its second ALT taken alone is
C>CAG, so the cell reads False. A locus with no coordinate or no ALT has nothing to judge.
Source code in enricher/src/just_dna_enricher/sequences.py
normalize_locus ¶
normalize_locus(
locus: Mapping[str, object],
read_window: Callable[[str, int, int], str | None]
| None,
) -> NormalizedLocus
Left-normalize a locus the enricher fills (RM270).
Each ALT takes its leftmost parsimonious spelling (left_normalize); ALTs that land on one
(start, ref) stay one locus, in their recorded order, and ALTs that land on two become two loci,
the split a caller's bcftools norm -m -any makes. Ensembl's X:634690 AGAG>AG,AGAGAG (SHOX) is a
2 bp deletion and a duplication that a caller writes X:634689 CAG>C and X:634689 C>CAG: two
references, so two loci. A spelling no ALT changes comes back as it was.
Whole or not at all. When any ALT cannot be decided (no reference read, a non-nucleotide
allele) the locus comes back unchanged with None: half a respelling would put two conventions in
one cell. Every other key of the locus is carried onto each piece.
Source code in enricher/src/just_dna_enricher/sequences.py
indel_spellings ¶
indel_spellings(
chrom: str,
pos: int,
ref: str,
alt: str,
read_window: Callable[[str, int, int], str | None],
) -> list[tuple[str, int, str, str]] | None
Every anchored VCF spelling of one pure indel inside its repeat, leftmost first, or None.
Left-align, then slide the event right one base at a time while the reference allows it: a
deletion of P over [pos+1, pos+n] also reads as a deletion one base right when the base after
it equals P[0], and its payload rotates to P[1:] + base; an insertion slides the same way
against the bases after its anchor. Each slide's new anchor base is that next base, so the spelling
one step right is base + payload against base. None for anything that is not a single-anchored
insertion or deletion, for an unreadable reference, and for a repeat longer than the window, which
is withheld rather than truncated (RM274, RM375: the payloads and the anchors are one walk).
Source code in enricher/src/just_dna_enricher/sequences.py
summarize_ref_mismatches ¶
Group findings by cause: one line per diagnosis with a count and a few named rows.
A systematic mistake produces one finding per row — 56 lines for a 69-variant module, ~2,400 for a
large panel — and a wall that long buries every other thing the run reported. Grouping by reason
rather than by row is what makes the shared cause visible, which is the whole point: 2,400 rows
disagreeing on ref reads as hopeless, while "2,400 rows are shifted one base right" is a
one-line fix.
Source code in enricher/src/just_dna_enricher/sequences.py
verify_reference_alleles ¶
verify_reference_alleles(
rows: list[ResolutionRow],
*,
sequences: SequenceProxy | None = None,
offline: bool = False,
) -> RefCheck
Compare each row's authored ref against the reference sequence. Returns the disagreements.
Skipped (an empty RefCheck carrying its reason) when there is no sequence access — offline, or
an unreachable service. A check that cannot run is not a check that passed, but it is also not a
failure: the rest of the enrichment is unaffected, and the run says it was skipped.
Rows without a coordinate, and rows whose ref is not plain ACGT (a symbolic or structural allele,
RM5), are not checked — there is nothing to compare, and inventing a verdict would be worse than
abstaining. Nor is a row whose read came back empty: the service answered nothing about that
locus, so it is outside subjects rather than inside it with a clean bill. Reads are deduplicated
through SequenceProxy's cache, so a module asking about one locus repeatedly costs one round trip.
A build with no refget table stops the whole pass, and says unsupported. Every row of a
genome_build: GRCh37 module raises UnsupportedBuildError at refget_accession, and that used
to be caught per row and continued — a skip indistinguishable from "this row had no coordinate",
leaving not_checked at None and the pass reporting as having run over zero subjects. The
attestation then published reference_allele: subjects 0, findings 0, skipped null for a module
nothing was compared on, and genome_build_agreement took its clean branch and stated "no
authored ref disagreed with the reference" about a comparison that never happened — the exact
self-contradiction _verification_records guards two of the three routes to. This is the third
route, and VALID_VERIFICATION_SKIPS already named it: unsupported, "this tier cannot put the
question for these rows (e.g. an unbuilt assembly)", which until now nothing emitted.
Decided from the module's build rather than by counting per-row failures, because the two are
different situations: an off-assembly contig on GRCh38 (refget_accession returns None) is
one row this tier has no sequence for and belongs outside subjects exactly as it is today, while
an unbuilt assembly means the question cannot be put for any row in the module.
Source code in enricher/src/just_dna_enricher/sequences.py
502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 | |