Skip to content

just_dna_compiler.compiler.validate

just_dna_compiler.compiler.validate

validate_spec and the validation pass behind it, plus the overlay loader and its targets.

load_overlay

load_overlay(
    spec_dir: Path,
) -> tuple[list[OverrideRow], list[str], list[str]]

Read overrides.csv and answer with (rows, errors, warnings) (RM124).

Public since RM136, because the enricher needs it: an author who corrects a derived cell through the overlay must not go on being told the same finding by the tier that writes the file. Private, it would have been reached into the way load_spec was before S74, or — worse — reimplemented in the enricher, which is the drift the overlay's own design refuses.

One loader for both public entry points, because both have to read it and a second copy is where validate and compile learn to disagree — the parity rule this module keeps re-learning (@validate-refuses-all). Everything it reports is structural: the rows parse or they do not, the key is duplicated or it is not, a key group carries one operation or several. Nothing here consults a derived table, which is what keeps every finding identical on both laps of a round trip.

The body is load_authored_table's, the one every authored table outside the table kinds shares (RM357); the overlay's own part is its registry entry's row_checks, overlay_coherence_errors.

Source code in compiler/src/just_dna_compiler/compiler/validate.py
def load_overlay(spec_dir: Path) -> tuple[list[OverrideRow], list[str], list[str]]:
    """Read `overrides.csv` and answer with `(rows, errors, warnings)` (RM124).

    **Public since RM136**, because the enricher needs it: an author who corrects a derived cell
    through the overlay must not go on being told the same finding by the tier that writes the file.
    Private, it would have been reached into the way `load_spec` was before S74, or — worse —
    reimplemented in the enricher, which is the drift the overlay's own design refuses.

    One loader for both public entry points, because both have to read it and a second copy is where
    `validate` and `compile` learn to disagree — the parity rule this module keeps re-learning
    (`@validate-refuses-all`). Everything it reports is structural: the rows parse or they do not, the
    key is duplicated or it is not, a key group carries one operation or several. Nothing here
    consults a derived table, which is what keeps every finding identical on both laps of a round
    trip.

    The body is `load_authored_table`'s, the one every authored table outside the table kinds shares
    (RM357); the overlay's own part is its registry entry's `row_checks`, `overlay_coherence_errors`.
    """
    return load_authored_table(spec_dir, OVERRIDES_CSV)

validate_spec

validate_spec(
    spec_dir: Path,
    authority_keys: Iterable[str] | None = None,
    *,
    strict: bool = False,
    resolve_with_ensembl: bool = True,
    ba1_threshold: float = BA1_ALLELE_FREQUENCY_THRESHOLD,
) -> ValidationResult

Validate a module spec directory without producing output.

authority_keys (inject-only) is the set of consumer/registry-owned identity keys to strip from the authored module: block before validation — pass just_dna_format.normalize. IDENTITY_AUTHORITY_KEYS (or your own set) so a legacy spec carrying namespace:/owner:/ canonical_id: validates; the format applies none by default. Stripped keys are surfaced on .info. Everything else still trips extra="forbid".

strict mirrors compile_module's flag and exists for one reason: several checks are a mode ladder (warning in best_effort, error in strict), so without a mode here the pre-flight could not answer the question the author actually asked — the documented order is validate then compile --strict, and a modeless validate is a pre-flight for the other compile.

It mirrors compile_module's severities exactly, which is not the same as changing severity only — this said the latter, in both of the two docstrings carrying it, and it was false (RM218). Two findings are aggregates with no best_effort counterpart sentence: the unresolved-position refusal (strict compile: N variant(s) …) and build_disagreement_error. Their best_effort rung is a different sentence — the per-subject rsid_unresolved warning, which fires in both modes — so under strict the aggregate is genuinely added rather than promoted. The contract the two commands share is that validate(strict=x) and compile(strict=x) reach the same verdict, not that the two modes of validate differ by a severity column.

resolve_with_ensembl mirrors it for the same reason and is passed through by compile_module. The pre-flight applies the injected table to the positional 0.4 tables (RM43), and that decides whether their rows are reported as unjoinable — so a validate that ignored the master resolution switch would be more optimistic than the compile it precedes, which is the disagreement direction the parity rule exists to prevent.

ba1_threshold mirrors it too (RM367): the ACMG BA1 lint is warning-only, but the pre-flight owes the same findings as the compile it precedes, and this number decides which variants it names.

Stats include genes/categories as lists (filtering None) plus variant_count, gene_count, study_count, and the ClinVar quality counts (clinvar_count/pathogenic_count/benign_count) — the fields the manifest needs. See ValidationResult.stats for the full key contract.

warnings is the complete list, unchanged; carried names the subset an author cannot clear and warnings_summary counts them by code (RM131). A caller that wants to keep building on this run's findings — as compile_module and close_module do — calls _validate_spec instead, for the reason given there.

Source code in compiler/src/just_dna_compiler/compiler/validate.py
def validate_spec(
    spec_dir: Path,
    authority_keys: Iterable[str] | None = None,
    *,
    strict: bool = False,
    resolve_with_ensembl: bool = True,
    ba1_threshold: float = BA1_ALLELE_FREQUENCY_THRESHOLD,
) -> ValidationResult:
    """Validate a module spec directory without producing output.

    `authority_keys` (inject-only) is the set of consumer/registry-owned identity keys to strip from
    the authored `module:` block before validation — pass `just_dna_format.normalize.
    IDENTITY_AUTHORITY_KEYS` (or your own set) so a legacy spec carrying `namespace:`/`owner:`/
    `canonical_id:` validates; the format applies none by default. Stripped keys are surfaced on
    `.info`. Everything else still trips `extra="forbid"`.

    `strict` mirrors `compile_module`'s flag and exists for one reason: several checks are a **mode
    ladder** (warning in `best_effort`, error in `strict`), so without a mode here the pre-flight
    could not answer the question the author actually asked — the documented order is `validate` then
    `compile --strict`, and a modeless `validate` is a pre-flight for the *other* compile.

    **It mirrors `compile_module`'s severities exactly, which is not the same as changing severity
    only** — this said the latter, in both of the two docstrings carrying it, and it was false
    (RM218). Two findings are *aggregates* with no `best_effort` counterpart sentence: the
    unresolved-position refusal (`strict compile: N variant(s) …`) and `build_disagreement_error`.
    Their `best_effort` rung is a different sentence — the per-subject `rsid_unresolved` warning,
    which fires in **both** modes — so under `strict` the aggregate is genuinely *added* rather than
    promoted. The contract the two commands share is that **`validate(strict=x)` and
    `compile(strict=x)` reach the same verdict**, not that the two modes of `validate` differ by a
    severity column.

    `resolve_with_ensembl` mirrors it for the same reason and is passed through by `compile_module`.
    The pre-flight applies the injected table to the positional 0.4 tables (RM43), and that decides
    whether their rows are reported as unjoinable — so a `validate` that ignored the master resolution
    switch would be *more optimistic* than the compile it precedes, which is the disagreement
    direction the parity rule exists to prevent.

    `ba1_threshold` mirrors it too (RM367): the ACMG BA1 lint is warning-only, but the pre-flight owes
    the same findings as the compile it precedes, and this number decides which variants it names.

    Stats include `genes`/`categories` as lists (filtering None) plus `variant_count`,
    `gene_count`, `study_count`, and the ClinVar quality counts
    (`clinvar_count`/`pathogenic_count`/`benign_count`) — the fields the manifest needs. See
    `ValidationResult.stats` for the full key contract.

    `warnings` is the complete list, unchanged; `carried` names the subset an author cannot clear and
    `warnings_summary` counts them by code (RM131). A caller that wants to keep *building* on this
    run's findings — as `compile_module` and `close_module` do — calls `_validate_spec` instead, for
    the reason given there.
    """
    return _validate_spec(
        spec_dir,
        authority_keys,
        strict=strict,
        resolve_with_ensembl=resolve_with_ensembl,
        ba1_threshold=ba1_threshold,
    )[0]