Turning a clinical-significance comparison into a record an author can act on (0.7, RM130).
The ClinVar cross-check has always known which rows disagree and has always thrown them away: it
reports twenty of 141,616 and the twenty reach a logger. This module is the other half — the two
verdicts, the per-authority calls behind them, and the writer that puts both beside the spec.
Two functions, and the split is the design. classify_concordance is a pure function over an
authored call and N authority outcomes; concordance_tables turns a run's findings into rows. The
classifier knows nothing about ClinVar, PubMind or any other source, which is what lets it be
exercised at three and five authorities today rather than at whatever N the current producer happens
to reach — and the arity of the vocabularies is the property the whole two-field split exists to
hold.
Nothing here resolves a split. With five authorities in a two-against-three disagreement,
precedence and majority pick different winners and choosing between them needs a weighting model.
This workspace has declined to invent one three times, so authored_position is a relation to the
set: computable with no weights, true at any topology, and leaving a consumer free to apply its own
model to the detail rows.
What contests a subject, stated once because it is the contract with the shipped check. A row is
written when a disagreement is established:
- the module's own call and an authority's call sit in opposite camps — a pathogenic-class call
against a benign-class one; or
- two authorities that spoke disagree with each other.
At one authority the second clause cannot fire, so the emitted set is exactly the conflicts the
shipped check already reports — which is what keeps the three-way check a superset of the two-way
rather than a second opinion beside it, and keeps an author from meeting one disagreement twice.
An authority that could not be consulted never contests anything on its own. unknown AND false
is false under Kleene, so an unreachable archive does not un-see a disagreement already witnessed;
but on its own it produces no row, because a question nobody could put is not a finding.
AuthorityCall
dataclass
AuthorityCall(
authority: str,
status: str,
clin_sig: str | None = None,
clin_sig_raw: str | None = None,
confidence: str | None = None,
confidence_unit: str | None = None,
dataset: str | None = None,
)
What one annotation authority had to say about one subject.
The classifier's input unit, and deliberately not tied to any source: it carries a normalized
clin_sig, the raw token behind it, and a confidence in the authority's own units with the
name of the instrument beside it. Nothing converts one authority's confidence into another's —
a gold-star count and an evidence-depth count are not the same quantity, and folding them into
one number would put three axes in one field.
spoke
property
Whether this authority actually stated a classification.
unreachable
property
Whether this authority could not be consulted at all — never an absence of a record.
ConcordanceVerdict
dataclass
ConcordanceVerdict(
authority_concordance: str,
authored_position: str,
opposed: bool | None,
contested: bool,
)
The two orthogonal verdicts, plus what follows from them.
contested is on the verdict rather than recomputed by the caller because it depends on facts
only the classifier holds — whether any authority actually spoke, and whether the camps in play
are opinionated. A caller re-deriving it from the two vocabulary members alone would call a
subject contested where nobody has a record, which is the vacuous-truth trap in the ∀ over an
empty set.
ConcordanceSubject
dataclass
ConcordanceSubject(
variant_key: str,
genotype: str,
authored_clin_sig: str | None,
calls: tuple[AuthorityCall, ...],
)
One (variant_key, genotype) the check put a question about, and every answer it got.
CallShift
dataclass
CallShift(
before: ClinSigAuthorityCallRow,
after: ClinSigAuthorityCallRow,
)
One authority's call about one answered subject, as it was recorded and as it reads now.
Both sides are whole ClinSigAuthorityCallRows rather than the two classifications, because the
dataset pair is what separates the two readings a reader actually needs: a re-released
archive that revised its call, and an archive that revised in place within one release. The
check does not branch on that difference — it has no rule for a dataset neither side recorded —
it just puts both in the message and lets the reader see which happened.
subject
property
The (variant_key, genotype) the answer was written about.
wording_held
property
Whether the authority's own words are unchanged while our reading of them moved.
True only when both sides recorded a verbatim token and the two are equal. That is this
release's normalizer having moved, not the archive — a different finding wearing the same
shape, and reporting it as an archive revision would accuse a source of a change we made.
AnsweredCallReport
dataclass
AnsweredCallReport(
shifts: tuple[CallShift, ...],
normalization: tuple[CallShift, ...],
withheld: tuple[tuple[str, str, str, str], ...],
subjects: int,
answered: int,
)
What a run can say about the answers this module already carries.
Nothing here is computed and discarded (@dont-discard-computed): answered and subjects
are the two denominators, and a caller that recomputed either would be re-implementing the
answered-subject rule — the drift the overlay's own design refuses.
moved_subjects
property
Answered subjects carrying at least one moved call. The numerator.
camp_of
camp_of(clin_sig: str | None) -> str | None
The camp a classification falls in, or None when there is no classification to place.
None rather than undecided for an absent value, because the two are different statements: a
source that said nothing has not landed in the undecided camp, it has not spoken at all. A
classification this release does not model reads as undecided — it was stated, and it opposes
nothing.
Source code in enricher/src/just_dna_enricher/concordance.py
| def camp_of(clin_sig: str | None) -> str | None:
"""The camp a classification falls in, or `None` when there is no classification to place.
`None` rather than `undecided` for an absent value, because the two are different statements: a
source that said nothing has not landed in the undecided camp, it has not spoken at all. A
classification this release does not model reads as `undecided` — it was stated, and it opposes
nothing.
"""
if clin_sig is None:
return None
return CLIN_SIG_CAMP.get(clin_sig, "undecided")
|
classify_concordance
classify_concordance(
authored_clin_sig: str | None,
calls: Sequence[AuthorityCall],
) -> ConcordanceVerdict
The two verdicts for one subject, at any number of authorities.
authority_concordance — do the authorities agree with each other?
A disagreement already witnessed is not un-witnessed by an authority that could not be asked
(unknown AND false is false), so discordant wins over unchecked. Otherwise an unreachable
authority leaves the question open, and the answer is unchecked rather than an agreement nobody
established. With everyone reachable it is concordant for two or more agreeing, single for
exactly one opinion — one voice is not corroboration — and none when every archive was asked
and none has a record.
authored_position — where does the module's own call sit relative to them?
matches_some is establishable under an unreachable sibling, because both halves of it are
witnessed by authorities that did speak. matches_all and matches_none are ∀ claims and need
the set closed, so an unreachable authority sends them to unchecked. absent is the module
stating no clinical call at all, which is a different thing from disagreeing with everyone.
Both are at camp granularity, the same coarseness the two-way check uses: pathogenic and
likely_pathogenic are one position, not two.
Source code in enricher/src/just_dna_enricher/concordance.py
| def classify_concordance(authored_clin_sig: str | None, calls: Sequence[AuthorityCall]) -> ConcordanceVerdict:
"""The two verdicts for one subject, at any number of authorities.
**`authority_concordance` — do the authorities agree with each other?**
A disagreement already witnessed is not un-witnessed by an authority that could not be asked
(`unknown AND false` is `false`), so `discordant` wins over `unchecked`. Otherwise an unreachable
authority leaves the question open, and the answer is `unchecked` rather than an agreement nobody
established. With everyone reachable it is `concordant` for two or more agreeing, `single` for
exactly one opinion — one voice is not corroboration — and `none` when every archive was asked
and none has a record.
**`authored_position` — where does the module's own call sit relative to them?**
`matches_some` is establishable under an unreachable sibling, because both halves of it are
witnessed by authorities that did speak. `matches_all` and `matches_none` are ∀ claims and need
the set closed, so an unreachable authority sends them to `unchecked`. `absent` is the module
stating no clinical call at all, which is a different thing from disagreeing with everyone.
Both are at **camp** granularity, the same coarseness the two-way check uses: `pathogenic` and
`likely_pathogenic` are one position, not two.
"""
spoke = [call for call in calls if call.spoke]
unreachable = [call for call in calls if call.unreachable]
spoken_camps = [camp_of(call.clin_sig) for call in spoke]
distinct_camps = set(spoken_camps)
if len(distinct_camps) > 1:
concordance = "discordant"
elif unreachable:
concordance = "unchecked"
elif len(spoke) >= 2:
concordance = "concordant"
elif len(spoke) == 1:
concordance = "single"
else:
concordance = "none"
authored_camp = camp_of(authored_clin_sig)
if authored_camp is None:
position = "absent"
else:
matched = [camp for camp in spoken_camps if camp == authored_camp]
missed = [camp for camp in spoken_camps if camp != authored_camp]
if matched and missed:
position = "matches_some"
elif unreachable:
position = "unchecked"
elif not spoke:
# Nobody has a record, so the ∀ is vacuous both ways. `matches_none` rather than
# `matches_all`, because "every authority agrees" must not be sayable about no authority
# at all — and this pairs with `none`, where the archives were asked and had nothing.
position = "matches_none"
elif not missed:
position = "matches_all"
else:
position = "matches_none"
# Opposed is a statement about the camps in play, the module's own included: two of them sit on
# opposite sides of the pathogenic/benign line. Establishable regardless of an unreachable
# authority — one more opinion cannot unmake an opposition already present.
camps_in_play = {camp for camp in spoken_camps if camp is not None}
if authored_camp is not None:
camps_in_play.add(authored_camp)
opposed_established = camps_in_play >= OPINIONATED_CAMPS
opposed: bool | None = True if opposed_established else (None if unreachable else False)
# An established disagreement, and only that, contests a subject. Two clauses: the module against
# an authority, which needs both sides opinionated because an uncertain call opposes nothing; and
# one authority against another, which cannot fire below two speakers. At one authority the
# second is unreachable, so this reduces exactly to what the shipped two-way check reports.
authored_contested = authored_camp in OPINIONATED_CAMPS and any(
camp in OPINIONATED_CAMPS and camp != authored_camp for camp in spoken_camps
)
contested = authored_contested or concordance == "discordant"
return ConcordanceVerdict(
authority_concordance=concordance,
authored_position=position,
opposed=opposed,
contested=contested,
)
|
concordance_tables
concordance_tables(
subjects: Sequence[ConcordanceSubject],
*,
checked_at: str | None = None,
) -> tuple[
list[ClinSigConcordanceRow],
list[ClinSigAuthorityCallRow],
]
The two tables for a run: contested subjects, and the calls behind each one.
Only contested subjects reach the record, and the detail rows follow the parent rather than
being written for every subject asked. A record of every agreement would be a copy of the
module's own clin_sig column with a second opinion attached, and the count of subjects
compared is already published as the check's denominator — recording it a second time as rows
would be the same number in two places.
Order is the subject order the caller supplied, and within a subject the call order it supplied,
both preserved rather than sorted: parquet bytes depend on row order, and a sort over values
would let a corrected classification move a row.
Source code in enricher/src/just_dna_enricher/concordance.py
| def concordance_tables(
subjects: Sequence[ConcordanceSubject], *, checked_at: str | None = None
) -> tuple[list[ClinSigConcordanceRow], list[ClinSigAuthorityCallRow]]:
"""The two tables for a run: contested subjects, and the calls behind each one.
**Only contested subjects reach the record**, and the detail rows follow the parent rather than
being written for every subject asked. A record of every agreement would be a copy of the
module's own `clin_sig` column with a second opinion attached, and the count of subjects
compared is already published as the check's denominator — recording it a second time as rows
would be the same number in two places.
Order is the subject order the caller supplied, and within a subject the call order it supplied,
both preserved rather than sorted: parquet bytes depend on row order, and a sort over values
would let a corrected classification move a row.
"""
parents: list[ClinSigConcordanceRow] = []
calls: list[ClinSigAuthorityCallRow] = []
for subject in subjects:
verdict = classify_concordance(subject.authored_clin_sig, subject.calls)
if not verdict.contested:
continue
parents.append(
ClinSigConcordanceRow(
variant_key=subject.variant_key,
genotype=subject.genotype,
authored_clin_sig=subject.authored_clin_sig,
authority_concordance=verdict.authority_concordance,
authored_position=verdict.authored_position,
opposed=verdict.opposed,
checked_at=checked_at,
)
)
calls.extend(
ClinSigAuthorityCallRow(
variant_key=subject.variant_key,
genotype=subject.genotype,
authority=call.authority,
status=call.status,
clin_sig=call.clin_sig,
clin_sig_raw=call.clin_sig_raw,
confidence=call.confidence,
confidence_unit=call.confidence_unit,
dataset=call.dataset,
checked_at=checked_at,
)
for call in subject.calls
)
return parents, calls
|
write_concordance_tables
write_concordance_tables(
spec_dir: Path,
parents: Sequence[ClinSigConcordanceRow],
calls: Sequence[ClinSigAuthorityCallRow],
*,
error: type[Exception] = ValueError,
) -> tuple[Path, ...]
Replace both tables beside the spec, and return the paths actually written.
error is the caller's own type for a licence table that will not merge (RM379), so a failure
surfaces as enrich's error rather than as a bare ValueError.
These are pure build products, and they are rewritten whole rather than merged. Every other
derived sidecar gap-fills because a recorded row might carry a curator's judgement; this one
cannot, because the judgement about a contested subject goes in overrides.csv and never into
this file. Merging would be actively wrong: a subject the archive stopped contesting has to
leave the record, since a conflict that stops being reported is exactly how an author learns
the archive caught up with them.
A run that found nothing contested therefore empties a record already on disk rather than leaving
the previous run's rows in place. Where there is no record yet it writes none (RM370,
layout.nothing_to_write): a header-only table beside a module with no clin_sig cell said
nothing, and the compiler already reads an empty record and an absent one alike (no manifest
block). How many subjects were compared is the check's denominator in verification.json, which
is where asked and found nothing contested is recorded; a run that could not put the question
returns no record at all and leaves the previous one exactly as it was.
One exception, and it is the vindication signal (RM117). A module whose overrides.csv
answers this record refers to it, so an empty record there is not about nothing: it is what turns
the answer into overlay_answer_vindicated (the authorities now agree) at compile time, where a
missing table would read overlay_targets_missing_table and tell the author to run the pass that
just ran. That arises when an author deleted the sidecar to regenerate it (@sidecar-authoritative).
Resolved through layout so a module carrying its sidecars under derived/ gets them written
where it keeps the rest, rather than growing a second copy at the root.
Source code in enricher/src/just_dna_enricher/concordance.py
| def write_concordance_tables(
spec_dir: Path,
parents: Sequence[ClinSigConcordanceRow],
calls: Sequence[ClinSigAuthorityCallRow],
*,
error: type[Exception] = ValueError,
) -> tuple[Path, ...]:
"""Replace both tables beside the spec, and return the paths actually written.
`error` is the caller's own type for a licence table that will not merge (RM379), so a failure
surfaces as `enrich`'s error rather than as a bare `ValueError`.
**These are pure build products, and they are rewritten whole rather than merged.** Every other
derived sidecar gap-fills because a recorded row might carry a curator's judgement; this one
cannot, because the judgement about a contested subject goes in `overrides.csv` and never into
this file. Merging would be actively wrong: a subject the archive stopped contesting has to
*leave* the record, since a conflict that stops being reported is exactly how an author learns
the archive caught up with them.
A run that found nothing contested therefore empties a record already on disk rather than leaving
the previous run's rows in place. Where there is no record yet it writes none (RM370,
`layout.nothing_to_write`): a header-only table beside a module with no `clin_sig` cell said
nothing, and the compiler already reads an empty record and an absent one alike (no manifest
block). How many subjects were compared is the check's denominator in `verification.json`, which
is where *asked and found nothing contested* is recorded; a run that could not put the question
returns no record at all and leaves the previous one exactly as it was.
**One exception, and it is the vindication signal (RM117).** A module whose `overrides.csv`
answers this record refers to it, so an empty record there is not about nothing: it is what turns
the answer into `overlay_answer_vindicated` (*the authorities now agree*) at compile time, where a
missing table would read `overlay_targets_missing_table` and tell the author to run the pass that
just ran. That arises when an author deleted the sidecar to regenerate it (`@sidecar-authoritative`).
Resolved through `layout` so a module carrying its sidecars under `derived/` gets them written
where it keeps the rest, rather than growing a second copy at the root.
"""
parent_path = sidecar_write_path(spec_dir, CONCORDANCE_CSV)
calls_path = sidecar_write_path(spec_dir, AUTHORITY_CALLS_CSV)
answered = bool(overlay_answered_subjects(spec_dir, CONCORDANCE_CSV))
# RM379, `@write-the-sourcerow`: an authority a call row names was consulted unless the row says
# `unchecked`, and each consulted one has its licence row, inside the calls table's commit
# (`@licence-row-inside-the-commit`), labelled by the release every such row states (RM373). One
# nobody asked contributes nothing and writes none; CIViC writes no call, so it writes none either.
consulted: dict[str, list[str | None]] = {}
for call in calls:
if call.status != "unchecked":
consulted.setdefault(call.authority, []).append(stated_release(call.dataset))
# Read before the write, so a placeholder licence table refuses before a table lands (S98, RM231).
require_sources_file(spec_dir, error=error)
parents_written = _write_table(parents, _CONCORDANCE_FIELDNAMES, parent_path, answered=answered)
calls_written = _write_table(
calls,
_AUTHORITY_CALL_FIELDNAMES,
calls_path,
answered=answered,
# Rewritten whole on every run that writes it, so the commit is the record's whole answer and an
# authority no consulted row names any more has its row retired with it (RM383). Handed over even
# with nothing consulted: a record that empties is the commit whose stale rows remain. A record
# that writes nothing at all (RM370) runs no commit, so it retires nothing.
before_commit=lambda: record_table_sources(
consulted, AUTHORITY_CALL_LAYER, spec_dir, error=error, answered_every_row=True
),
)
written = tuple(
path for path, wrote in ((parent_path, parents_written), (calls_path, calls_written)) if wrote
)
if written:
logger.info(
"Wrote the clinical-significance concordance record: %d contested subject(s), %d authority "
"call(s). Answer one with an overrides.csv row against %s.",
len(parents),
len(calls),
CONCORDANCE_CSV,
)
return written
|
read_recorded_calls
read_recorded_calls(
spec_dir: Path,
) -> list[ClinSigAuthorityCallRow] | None
The authority calls this module already has on file, or None when it has none (RM151).
The baseline half of the staleness comparison, and it must be read before the commit rewrites
it. write_concordance_tables replaces this table whole, so the previous run's rows are the
record as it stood when an author read it and wrote their answer — available for exactly as long
as this run has not yet committed. enrich() computes everything before its commit already, for
the unrelated reason that a refused strict run must change nothing, and that ordering is what
makes this readable at all.
None for a module with no record on disk, [] for one whose record is empty. Since RM370 no
record is also what a run that asked and found nothing contested leaves where there was none
before (layout.nothing_to_write); answered_call_shift reads the two alike — every lookup
misses, and an answered call is withheld as no_prior_record either way.
A file that will not parse is None too, and says so. Guessing at a half-read baseline would
manufacture a movement out of our own inability to read, which is the one wrong answer available.
Resolved through layout so a module keeping its sidecars under derived/ is read where it
keeps them (@sidecar-name-and-place).
Source code in enricher/src/just_dna_enricher/concordance.py
| def read_recorded_calls(spec_dir: Path) -> list[ClinSigAuthorityCallRow] | None:
"""The authority calls this module already has on file, or `None` when it has none (RM151).
**The baseline half of the staleness comparison, and it must be read before the commit rewrites
it.** `write_concordance_tables` replaces this table whole, so the previous run's rows are the
record as it stood when an author read it and wrote their answer — available for exactly as long
as this run has not yet committed. `enrich()` computes everything before its commit already, for
the unrelated reason that a refused `strict` run must change nothing, and that ordering is what
makes this readable at all.
`None` for a module with no record on disk, `[]` for one whose record is empty. Since RM370 no
record is also what a run that asked and found nothing contested leaves where there was none
before (`layout.nothing_to_write`); `answered_call_shift` reads the two alike — every lookup
misses, and an answered call is withheld as `no_prior_record` either way.
A file that will not parse is `None` too, and says so. Guessing at a half-read baseline would
manufacture a movement out of our own inability to read, which is the one wrong answer available.
Resolved through `layout` so a module keeping its sidecars under `derived/` is read where it
keeps them (`@sidecar-name-and-place`).
"""
path = resolve_sidecar(spec_dir, AUTHORITY_CALLS_CSV)
if path is None:
return None
rows, errors, _ = load_csv_rows(path, ClinSigAuthorityCallRow, path.name)
if errors:
logger.warning(
"%s is present but unreadable (%s); no answered call can be compared against it this "
"run, which is reported as unknown rather than as unchanged.",
path.name,
errors[0],
)
return None
return rows
|
shifted_authority_calls
shifted_authority_calls(
baseline: Sequence[ClinSigAuthorityCallRow] | None,
fresh: Sequence[ClinSigAuthorityCallRow] | None,
answered: Sequence[tuple[str, str]],
) -> AnsweredCallReport
The answered subjects whose authority calls have moved since the answer was written (RM151).
An overrides.csv row against the concordance record is a judgement about a particular
disagreement — the archive said X, the author says Y, and the reason column explains why. If
the archive later says Z, that reason was written about a value that is no longer there. Nothing
in the record distinguishes a justification that still describes the disagreement on file from
one that describes a disagreement since replaced by a different one, and this is the check that
notices.
The comparison is recorded-call against fresh-call, per (variant_key, genotype, authority),
and the previous clin_sig_authority_calls.csv is the only baseline this format has. That
table is what each authority actually said, with clin_sig, the verbatim clin_sig_raw and the
dataset it came from — so the comparison is available here and nowhere else. Do not promise
this for a table that records no prior value (@probe-names-the-table): an overlay row against
frequencies.csv or resolution.csv has no recorded baseline at all, so a general "the value
moved" check would be answerable for one table and silently absent for the rest.
It observes, it does not adjudicate. The disagreement you answered is not the disagreement
that exists now is a statement about the record. Your answer may be wrong is a verdict, and
this format does not put a verdict under a check that cannot see the reasoning — the same
restraint the concordance tables keep everywhere else.
A subject that left the record entirely is not this finding. It is not in fresh, so it
never enters the loop: the authorities stopped contesting it, which is RM117's
overlay_answer_vindicated and is reported by the compiler as good news. Two findings about one
overlay row would be one of them firing with the wrong words on it.
Order is fresh row order, preserved rather than sorted: the message is read by a human and a
sort over values would let a corrected classification move a line.
Source code in enricher/src/just_dna_enricher/concordance.py
| def shifted_authority_calls(
baseline: Sequence[ClinSigAuthorityCallRow] | None,
fresh: Sequence[ClinSigAuthorityCallRow] | None,
answered: Sequence[tuple[str, str]],
) -> AnsweredCallReport:
"""The answered subjects whose authority calls have moved since the answer was written (RM151).
**An `overrides.csv` row against the concordance record is a judgement about a particular
disagreement** — the archive said X, the author says Y, and the `reason` column explains why. If
the archive later says Z, that reason was written about a value that is no longer there. Nothing
in the record distinguishes a justification that still describes the disagreement on file from
one that describes a disagreement since replaced by a different one, and this is the check that
notices.
**The comparison is recorded-call against fresh-call, per `(variant_key, genotype, authority)`,
and the previous `clin_sig_authority_calls.csv` is the only baseline this format has.** That
table is what each authority actually said, with `clin_sig`, the verbatim `clin_sig_raw` and the
`dataset` it came from — so the comparison is available here and nowhere else. **Do not promise
this for a table that records no prior value** (`@probe-names-the-table`): an overlay row against
`frequencies.csv` or `resolution.csv` has no recorded baseline at all, so a general "the value
moved" check would be answerable for one table and silently absent for the rest.
**It observes, it does not adjudicate.** *The disagreement you answered is not the disagreement
that exists now* is a statement about the record. *Your answer may be wrong* is a verdict, and
this format does not put a verdict under a check that cannot see the reasoning — the same
restraint the concordance tables keep everywhere else.
**A subject that left the record entirely is not this finding.** It is not in `fresh`, so it
never enters the loop: the authorities stopped contesting it, which is RM117's
`overlay_answer_vindicated` and is reported by the compiler as good news. Two findings about one
overlay row would be one of them firing with the wrong words on it.
Order is `fresh` row order, preserved rather than sorted: the message is read by a human and a
sort over values would let a corrected classification move a line.
"""
answered_keys = set(answered)
if fresh is None:
# No run to compare against — nobody could be consulted, so there is not even a set of
# authorities to name. Zero comparable subjects, which is not the claim that nothing moved;
# `answered` carries the gap and the notes are what say it out loud.
return AnsweredCallReport((), (), (), 0, len(answered_keys))
# **A missing baseline is an empty one, not a short circuit.** `None` here means the module has
# never had a record written, or the one it has will not parse; either way every lookup below
# misses and every answered call is withheld as `no_prior_record` — which is the reading, and it
# is worth saying per authority rather than collapsing into "the question could not be put". The
# first run to write a record takes this path, and its note is the useful one: run once more.
before = {(row.variant_key, row.genotype, row.authority): row for row in (baseline or ())}
shifts: list[CallShift] = []
normalization: list[CallShift] = []
withheld: list[tuple[str, str, str, str]] = []
comparable: set[tuple[str, str]] = set()
for now in fresh:
subject = (now.variant_key, now.genotype)
if subject not in answered_keys:
continue
key = (now.variant_key, now.genotype, now.authority)
then = before.get(key)
if then is None or then.status == "unchecked":
withheld.append((*key, "no_prior_record"))
continue
if now.status == "unchecked":
withheld.append((*key, "unchecked_now"))
continue
comparable.add(subject)
if then.status == now.status and then.clin_sig == now.clin_sig:
# Unchanged, `dataset` included or not: a re-released archive that says the same thing
# has not moved the disagreement the answer was written about.
continue
shift = CallShift(before=then, after=now)
(normalization if shift.wording_held else shifts).append(shift)
return AnsweredCallReport(
shifts=tuple(shifts),
normalization=tuple(normalization),
withheld=tuple(withheld),
subjects=len(comparable),
answered=len(answered_keys),
)
|
answered_call_sentences
answered_call_sentences(
report: AnsweredCallReport,
) -> list[str]
The findings a run reports about its answered subjects, warning-tier, in both modes (RM151).
Never escalated under strict (@clinsig-never-escalates), and with more force than for the
concordance record itself: nothing here is even a disagreement, it is a note that the record an
author reasoned over has been rewritten underneath their reasoning. Refusing a build over it
would gate an artifact on an archive's release schedule.
Two sentences, kept apart because they are about different parties. The first is the archive
revising; the second is this release's own normalizer reading unchanged wording differently,
which is a fact about our code and is not the author's to act on.
Empty when nothing moved and empty when nothing could be compared — a check that cannot fail
reports no zero (@tautology-zero), and the gap belongs in the notes rather than dressed up as a
clean bill.
Source code in enricher/src/just_dna_enricher/concordance.py
| def answered_call_sentences(report: AnsweredCallReport) -> list[str]:
"""The findings a run reports about its answered subjects, warning-tier, in both modes (RM151).
**Never escalated under `strict`** (`@clinsig-never-escalates`), and with more force than for the
concordance record itself: nothing here is even a disagreement, it is a note that the record an
author reasoned over has been rewritten underneath their reasoning. Refusing a build over it
would gate an artifact on an archive's release schedule.
Two sentences, kept apart because they are about different parties. The first is the archive
revising; the second is **this release's own normalizer** reading unchanged wording differently,
which is a fact about our code and is not the author's to act on.
Empty when nothing moved and empty when nothing could be compared — a check that cannot fail
reports no zero (`@tautology-zero`), and the gap belongs in the notes rather than dressed up as a
clean bill.
"""
lines: list[str] = []
if report.shifts:
lines.append(
f"{report.moved_subjects} of {report.subjects} answered subject(s) rest on an authority "
f"call that has moved since the answer was recorded: {_render_shifts(report.shifts)}. "
f"The overrides.csv reason for each was written about the earlier call, so it describes "
f"a disagreement that is no longer the one on record. Nothing here grades the answer — "
f"re-read the reason and decide whether it still says what you mean. Asked for "
f"{CONCORDANCE_CSV} alone: it is the one table this format keeps a prior value in."
)
if report.normalization:
lines.append(
f"{len(report.normalization)} answered call(s) differ only after this release's own "
f"normalization — the authority's verbatim wording is unchanged: "
f"{_render_shifts(report.normalization)}. What moved is how this tier reads the token, "
f"not what the archive published, so the answer is about the same disagreement it "
f"always was and there is nothing for an author to do."
)
return lines
|
answered_call_notes
answered_call_notes(
report: AnsweredCallReport,
) -> list[str]
What a run should say about answers it could not put the question for. Info-tier.
Said out loud rather than left silent, because a comparison that quietly did not run reads as one
that found nothing — the failure the whole tri-state exists to prevent
(@unreachable-not-absent). Grouped by reason rather than by row, like every repeated finding
here.
Silent for a module with no answers at all, which is every module today: a check that cannot fire
must not announce a zero.
Source code in enricher/src/just_dna_enricher/concordance.py
| def answered_call_notes(report: AnsweredCallReport) -> list[str]:
"""What a run should say about answers it could **not** put the question for. Info-tier.
Said out loud rather than left silent, because a comparison that quietly did not run reads as one
that found nothing — the failure the whole tri-state exists to prevent
(`@unreachable-not-absent`). Grouped by reason rather than by row, like every repeated finding
here.
Silent for a module with no answers at all, which is every module today: a check that cannot fire
must not announce a zero.
"""
if not report.answered:
return []
notes: list[str] = []
uncomparable = report.answered - report.subjects
if uncomparable:
notes.append(
f"{uncomparable} of {report.answered} answered subject(s) could not be compared against "
f"what the authorities said when the answer was written. That is not agreement and not "
f"an absence of movement: it is a question nobody could put."
)
reasons: dict[str, int] = {}
for *_key, reason in report.withheld:
reasons[reason] = reasons.get(reason, 0) + 1
if reasons.get("no_prior_record"):
notes.append(
f"{reasons['no_prior_record']} answered call(s) have no prior {AUTHORITY_CALLS_CSV} row "
f"to compare against — this run is writing the first record, the previous one would not "
f"parse, or the authority was unchecked when the answer was written. Once a record is on "
f"file the comparison becomes available on the next run."
)
if reasons.get("unchecked_now"):
notes.append(
f"{reasons['unchecked_now']} answered call(s) come from an authority this run could not "
f"consult, so what it says today is unknown. Nothing is claimed about them either way."
)
return notes
|