just_dna_format.concordance

just_dna_format.concordance

The clinical-significance concordance record, in two paired tables (0.7, RM130).

A check counted its findings and kept none of them. enrich() compares each authored clin_sig against ClinVar's and reports twenty of 141,616 — a number an author can read and not act on, because nothing says which twenty in a form anything can join to. clin_sig_concordance.csv is those rows, named.

A conflict is a question, and an overrides.csv row is the answer. That sentence is the whole lifetime. The record is machine-written and never hand-edited: an author who has read a contested row and decided records the decision as an overlay row against this table — with the reason the overlay makes mandatory — rather than editing the finding away. Two things follow. The decision travels with the module instead of living in a curator's head. And the terminal state becomes visible for free: an overlay row that stops changing anything means the archive caught up with the author, which is evidence that a judgement was vindicated and is available nowhere else in this format.

overrides.csv, never provenance.json's outranks. Both record an authored value beating a source with prose, and 0.7 settled the overlap as a dated succession rather than a merge: the overlay wins and outranks is filed for removal at the major. Pointing new authors at the mechanism that survives costs a sentence, so this table's documentation and the warning that reports it both name the overlay and neither names the knob.

Why two tables rather than one

The agreement state belongs to the subject and each authority's words belong to the authority, and one row cannot hold both without either nesting a cell or keying on the authority. Keying on the authority gives N rows per variant and leaves the state with nowhere to live; nesting gives a cell a reader has to parse. So:

• clin_sig_concordance.csv, keyed (variant_key, genotype) — one row per contested subject, and the key is stable at any number of authorities. That stability is the point of the split: the earlier draft named its authority in a field (ClinSigConflict.clinvar), which would have cost a key change or a retype the moment a second authority arrived — major-only work, one item later.
• clin_sig_authority_calls.csv, keyed (variant_key, genotype, authority) — what each authority actually said, its raw token, and its confidence.

The key cannot be a bare variant_key. The comparison is of an authored call for a genotype, and annotations.parquet keys on genotype for the same reason; a table keyed on the variant alone collapses two authored calls that disagree with the archive differently.

Confidence is not normalized across authorities. ClinVar's review_stars and a literature miner's evidence-depth count are different instruments measuring different things, and folding them into one number is three axes in one field. So the detail row carries the value the authority published and the name of the instrument beside it, and nothing in this tier compares two of them.

What the record deliberately does not do

Nothing resolves a split. With five authorities in a two-against-three disagreement, a declared precedence order says one thing and a majority says another, and choosing between those rules is a judgement about how rank trades against agreement count — a weighting model. This workspace has declined to invent one three times, and the same refusal applies here: there is no majority column, no consensus call and no resolved winner. authored_position is a relation to the set, computable with no weights and true at any topology, and the detail rows carry everything a consumer with its own model needs.

And it never escalates. A disagreement with an archive is a fact about the field, not a defect in the module: half the time the archive is the stale side, and failing a build on one would have the format arbitrate a clinical dispute. Warning-tier in both modes, exactly like the check it records.

ClinSigConcordanceRow

Bases: BaseModel

One contested subject: how the authorities sit, and where the module's own call sits.

Standalone rather than an AuthoredModel, like every other machine-produced fact row: a human writes no row of this table, and extra="forbid" catches a typo'd column instead of dropping it.

The two verdicts are separate fields because they are separate questions (Principle 5), and that separation is what makes both vocabularies five members at two authorities and five at five. A single field would have to name the authority inside the member to say the same thing, which is the combinatorial explosion a stress test at five sources found.

ClinSigAuthorityCallRow

Bases: BaseModel

What one authority said about one subject, in that authority's own terms.

The detail half of the record, keyed (variant_key, genotype, authority) — so a subject carries one row per authority consulted, and adding an authority adds rows rather than columns.

It has no source column, and the omission is structural rather than an oversight. authority already names the annotation source this row came from, and a second column holding the same string would be two spellings of one fact — the overloading Principle 5 forbids. The consequence is that this table is exempt from the orphan check that asks which declared licence rows a module's tables actually use, which is correct: the pass that consulted the authority writes its own sources.csv row, and that is where the licence position is recorded.

An authority's words are not the author's to correct, which is why this table is outside the overlay's covered set while its parent is inside. The author answers the question; they do not get to rewrite what an archive published.

model_post_init

model_post_init(__context: object) -> None

Refuse a magnitude with no unit, and a call with no classification.

Two coherence rules the field validators cannot see on their own, both of them the same mistake in different clothes — a cell that reads as an answer and is not one.

confidence without confidence_unit is the weight lesson restated: nothing downstream can read 2 without being told it is a gold-star count, and the two authorities this record is built for publish numbers on the same scale that mean different things.

status='recorded' with no clin_sig claims an authority classified this subject while naming no classification, which is the shape an unknown takes when it is written down as an answer. no_record and unchecked are how a consultation with nothing to report is said.

Source code in schema/src/just_dna_format/concordance.py

def model_post_init(self, __context: object) -> None:
    """Refuse a magnitude with no unit, and a call with no classification.

    Two coherence rules the field validators cannot see on their own, both of them the same
    mistake in different clothes — a cell that reads as an answer and is not one.

    `confidence` without `confidence_unit` is the `weight` lesson restated: nothing downstream can
    read `2` without being told it is a gold-star count, and the two authorities this record is
    built for publish numbers on the same scale that mean different things.

    `status='recorded'` with no `clin_sig` claims an authority classified this subject while
    naming no classification, which is the shape an unknown takes when it is written down as an
    answer. `no_record` and `unchecked` are how a consultation with nothing to report is said.
    """
    if self.confidence is not None and self.confidence_unit is None:
        raise ValueError(
            f"confidence={self.confidence!r} names a magnitude with no instrument beside it — "
            f"set confidence_unit (e.g. 'review_stars'), because two authorities publish numbers "
            f"on scales that are not the same quantity"
        )
    if self.status == "recorded" and self.clin_sig is None:
        raise ValueError(
            "status='recorded' says this authority classified the subject, so clin_sig must "
            "carry the classification; an authority that was consulted and had nothing is "
            "'no_record', and one that could not be consulted at all is 'unchecked'"
        )
    if self.status != "recorded" and self.clin_sig is not None:
        raise ValueError(
            f"status={self.status!r} says this authority stated no classification, so clin_sig "
            f"must be empty — an unknown is withheld, never filled in with a value nobody gave"
        )