Skip to content

just_dna_compiler.compiler.fact_checks

just_dna_compiler.compiler.fact_checks

Checks over the derived fact tables: frequency and gene-metric arithmetic, literature, BA1, gene validity currency, and the cross-checks against variants.csv.

split_cited_literature

split_cited_literature(
    rows: list[LiteratureRow],
    studies: list[StudyRow],
    kind_rows: dict[str, list[Any]] | None = None,
) -> tuple[list[LiteratureRow], list[LiteratureRow]]

(kept, dropped) — the literature rows this module actually cites, and the rest (RM79).

The compiler discards the rest; literature.csv keeps them. A row describing a citation no study and no citing table row names is dead weight in the artifact: nothing joins to it, and it is only there because literature.csv is merge-not-clobber, so a citation the author has since deleted from studies.csv leaves its row behind. Keeping the row in the CSV is the point of that rule — it is the pin that makes a re-run cheap — and carrying it into the parquet and the manifest is a separate decision that nobody had taken deliberately.

What this settles. manifest.literature.missing_count counted exists is False over every row in the table while the citation_existence verification record counted over the module's current citations, so the two disagreed in a published manifest with nothing wrong in the module. Both were honest about their own subject, which is what made it a decision rather than a bug. Filtering here makes them the same subject by construction, rather than documenting a discrepancy a reader would have to reconcile.

cited empty means discard nothing, deliberately, and it is not the degenerate case it looks like: a module that cites nothing at all cannot distinguish "the sidecar is stale" from "the citations are not authored yet", and emptying its whole table on that reading would delete an enrichment pass's entire output. The if cited guard the orphan check already had is kept for the same reason it existed.

On the round trip. reverse_module rebuilds literature.csv from the parquet, so a reversed copy carries the kept rows only. That is a deterministic narrowing rather than a P7 breach — literature.csv is a machine-written derived sidecar, not an authored value (the RM69 reading of Principle 7's letter) — and it converges: everything in the parquet is cited by construction, so lap two discards nothing and the signatures are a fixed point. The rows are recoverable the way every derived sidecar's are, by re-running the pass.

Source code in compiler/src/just_dna_compiler/compiler/fact_checks.py
def split_cited_literature(
    rows: list[LiteratureRow],
    studies: list[StudyRow],
    kind_rows: dict[str, list[Any]] | None = None,
) -> tuple[list[LiteratureRow], list[LiteratureRow]]:
    """`(kept, dropped)` — the literature rows this module actually cites, and the rest (RM79).

    **The compiler discards the rest; `literature.csv` keeps them.** A row describing a citation no
    study and no citing table row names is dead weight in the artifact: nothing joins to it, and it
    is only there
    because `literature.csv` is merge-not-clobber, so a citation the author has since deleted from
    `studies.csv` leaves its row behind. Keeping the row in the CSV is the point of that rule — it is
    the pin that makes a re-run cheap — and carrying it into the parquet and the manifest is a
    separate decision that nobody had taken deliberately.

    **What this settles.** `manifest.literature.missing_count` counted `exists is False` over *every*
    row in the table while the `citation_existence` verification record counted over the module's
    *current* citations, so the two disagreed in a published manifest with nothing wrong in the
    module. Both were honest about their own subject, which is what made it a decision rather than a
    bug. Filtering here makes them the same subject **by construction**, rather than documenting a
    discrepancy a reader would have to reconcile.

    **`cited` empty means discard nothing**, deliberately, and it is not the degenerate case it looks
    like: a module that cites nothing at all cannot distinguish "the sidecar is stale" from "the
    citations are not authored yet", and emptying its whole table on that reading would delete an
    enrichment pass's entire output. The `if cited` guard the orphan check already had is kept for the
    same reason it existed.

    **On the round trip.** `reverse_module` rebuilds `literature.csv` from the parquet, so a reversed
    copy carries the kept rows only. That is a deterministic narrowing rather than a P7 breach —
    `literature.csv` is a machine-written derived sidecar, not an authored value (the RM69 reading of
    Principle 7's letter) — and it **converges**: everything in the parquet is cited by construction,
    so lap two discards nothing and the signatures are a fixed point. The rows are recoverable the way
    every derived sidecar's are, by re-running the pass.
    """
    cited = cited_pmids(studies, kind_rows)
    if not cited:
        return list(rows), []
    kept = [r for r in rows if r.pmid in cited]
    return kept, [r for r in rows if r.pmid not in cited]

cited_pmids

cited_pmids(
    studies: list[StudyRow],
    kind_rows: dict[str, list[Any]] | None = None,
) -> set[str]

Every PMID this module cites, from both citation sites, through the one normalizer.

Extracted so RM137's reachability predicate asks the same question split_cited_literature answers, rather than a second statement of it. The two would drift silently and in the worst direction: the predicate would call a row unreachable that the drop had kept, so a healthy overlay would report a finding forever.

The empty case is the caller's to interpret, and it is not "nothing is cited". A module citing nothing cannot distinguish a stale sidecar from citations not yet authored, so split_cited_literature discards nothing there — and the predicate must mirror that or every literature update on such a module reads as unreachable. literature_target_survives builds the mirror; nothing should test this set for emptiness on its own.

Source code in compiler/src/just_dna_compiler/compiler/fact_checks.py
def cited_pmids(studies: list[StudyRow], kind_rows: dict[str, list[Any]] | None = None) -> set[str]:
    """Every PMID this module cites, from both citation sites, through the one normalizer.

    Extracted so RM137's reachability predicate asks the **same** question `split_cited_literature`
    answers, rather than a second statement of it. The two would drift silently and in the worst
    direction: the predicate would call a row unreachable that the drop had kept, so a healthy overlay
    would report a finding forever.

    **The empty case is the caller's to interpret, and it is not "nothing is cited".** A module citing
    nothing cannot distinguish a stale sidecar from citations not yet authored, so `split_cited_literature`
    discards nothing there — and the predicate must mirror that or every literature `update` on such a
    module reads as unreachable. `literature_target_survives` builds the mirror; nothing should test
    this set for emptiness on its own.
    """
    cited: set[str] = set()
    for study in studies:
        cited.update(extract_pmids(study.pmid))
    cited.update(table_citations(kind_rows or {}))
    return cited

literature_target_survives

literature_target_survives(
    studies: list[StudyRow],
    kind_rows: dict[str, list[Any]] | None = None,
) -> Callable[[str], bool]

Can an artifact of this module carry a literature.csv row for this PMID? (RM137)

True when the PMID is cited, because split_cited_literature keeps exactly the cited rows — and True for everything when the module cites nothing at all, which is that function's own guard reproduced rather than restated. Without the guard a module with no citations would mark every literature correction unreachable: a stable false positive, which is worse than the unstable true one RM137 is about.

Source code in compiler/src/just_dna_compiler/compiler/fact_checks.py
def literature_target_survives(
    studies: list[StudyRow], kind_rows: dict[str, list[Any]] | None = None
) -> Callable[[str], bool]:
    """Can an artifact of this module carry a `literature.csv` row for this PMID? (RM137)

    True when the PMID is cited, because `split_cited_literature` keeps exactly the cited rows — and
    **True for everything when the module cites nothing at all**, which is that function's own guard
    reproduced rather than restated. Without the guard a module with no citations would mark every
    literature correction unreachable: a stable false positive, which is worse than the unstable true
    one RM137 is about.
    """
    cited = cited_pmids(studies, kind_rows)
    if not cited:
        return lambda pmid: True
    return lambda pmid: pmid in cited

resolution_target_survives

resolution_target_survives(
    subjects: ModuleSubjects,
    resolution_rows: list[ResolutionRow],
) -> Callable[[str], bool]

Can an artifact of this module carry a resolution.csv row for this variant_key? (RM137)

resolution.csv has no parquet, so reverse_module rebuilds it from the SNP core and _write_resolution_csv skips a row with no resolved position — "rows without a resolved position carry no fact and are skipped". So the surviving set is the subjects this module can place.

Computed from the authored coordinates and the injected table together, which is what makes it answer the same on both laps: on lap 1 the unpositioned row is present and its own cells say it is unpositioned; on lap 2 the row is gone and the authored side still says the same thing. Neither reading depends on the row being there to be matched.

The authored half is every locus row's, not variants.csv's alone (RM362). reverse_module writes a resolution.csv row for each placed row of a positional table too (RM43), so a heteroplasmy.csv row with authored coordinates was reported "no artifact of this module can carry" on lap 1 and matched on lap 2. subjects.placed is module_subjects' answer.

Source code in compiler/src/just_dna_compiler/compiler/fact_checks.py
def resolution_target_survives(
    subjects: ModuleSubjects, resolution_rows: list[ResolutionRow]
) -> Callable[[str], bool]:
    """Can an artifact of this module carry a `resolution.csv` row for this `variant_key`? (RM137)

    `resolution.csv` has no parquet, so `reverse_module` rebuilds it from the SNP core and
    `_write_resolution_csv` skips a row with no resolved position — *"rows without a resolved position
    carry no fact and are skipped"*. So the surviving set is the subjects this module can **place**.

    Computed from the authored coordinates and the injected table together, which is what makes it
    answer the same on both laps: on lap 1 the unpositioned row is present and its own cells say it is
    unpositioned; on lap 2 the row is gone and the authored side still says the same thing. Neither
    reading depends on the row being there to be matched.

    **The authored half is every locus row's, not `variants.csv`'s alone (RM362).** `reverse_module`
    writes a `resolution.csv` row for each placed row of a positional table too (RM43), so a
    `heteroplasmy.csv` row with authored coordinates was reported "no artifact of this module can
    carry" on lap 1 and matched on lap 2. `subjects.placed` is `module_subjects`' answer.
    """
    placed: set[str] = {row.variant_key for row in resolution_rows if row.chrom and row.start is not None}
    placed.update(subjects.placed)
    return lambda subject: subject in placed