Skip to content

just_dna_compiler.compiler.meta

just_dna_compiler.compiler.meta

The meta-table at compile (0.8): load meta.csv, expand every member, resolve precedence, and build meta.parquet and meta_resolved.parquet.

The model and the grammar are the format tier's (just_dna_format.meta); this module needs the module's other tables in hand, which is why it lives here. The table is loaded by the one loader every authored table outside the table kinds shares (load.load_authored_table, RM357).

Expansion reads the authored rows, after the symbolic-allele drop and before resolution, on both sides: validate_spec and compile_module hand it the same view, so a member matches the same rows in the pre-flight and in the compile and every finding is one sentence on both (@parity-by-check). For a variants.csv row the recorded key is the authored (variant_key, genotype), which is the key weights.parquet carries for every row resolution did not expand. An rsID that resolution expands onto several loci is the one residual: its weights rows are coordinate-keyed, and a consumer reaches them from the meta key through their rsid column (COMPILER.md § The meta-table).

What meta_resolved.parquet holds. One row per single-statement row a one-member meta row reaches, per (sex, drug) scope, with each fact column (tier, age_of_action, age_of_action_confidence, flags) after precedence and a withheld list naming the columns two incomparable, disagreeing rows left unresolved. A row of two or more members states facts about the combination, so its facts stay on its own meta.parquet row and never reach a member row by itself.

tradeoff (RM327) is computed there too, per allele-bearing member and never pooled across members: True when one member has a protective per-trait row and a risk per-trait row (a one-member row with trait_efo_id and a direction not withheld; a mediator row counts, since its direction is still a benefit/harm judgement), and that verdict lands on every row the member reaches, including the diplotypes an allele_function member reaches through the allele they carry. On one trait at one row, the fact columns' precedence applies first, so a narrower judgement the author made about that row (a diplotype member over the allele it contains, a genotype over its locus, a sex- or drug-scoped row over the unscoped one) drops the broader judgement on that trait. A row two members reach is True when either member is. Never False: RM327's false arm needs every trait's judgement stated, and a module states no closed list of the traits an allele touches, so absence of an opposing row is unknown, not a negative. Two alleles meeting at a heterozygous diplotype are two members, so ε2 protective on one trait and ε4 risk on another is not a trade-off of either.

MetaMatch

Bases: NamedTuple

One row a member matched: which member, which table, the row's position there and its key.

MetaBuild dataclass

MetaBuild(
    rows: list[MetaRow],
    members: list[list[MetaMember]],
    texts: list[list[str]],
    matches: list[list[MetaMatch]],
    withheld: list[list[str]],
    resolved: list[dict[str, Any]],
    errors: list[str],
    warnings: list[str],
)

Everything the compile writes from meta.csv, computed before anything is written.

selector_counts

selector_counts() -> list[MetaSelectorCount]

Every member of every row with its match count, in authored order, for the manifest.

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def selector_counts(self) -> list[MetaSelectorCount]:
    """Every member of every row with its match count, in authored order, for the manifest."""
    return [
        MetaSelectorCount(
            row=index, selector=text, matches=sum(1 for m in self.matches[index] if m.member == position)
        )
        for index, texts in enumerate(self.texts)
        for position, text in enumerate(texts)
    ]

member_tables

member_tables(
    variants: list[Any], kind_rows: Mapping[str, list[Any]]
) -> dict[str, list[Any]]

The rows each member table holds, keyed by the name a member uses (variants, not the file).

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def member_tables(variants: list[Any], kind_rows: Mapping[str, list[Any]]) -> dict[str, list[Any]]:
    """The rows each member table holds, keyed by the name a member uses (`variants`, not the file)."""
    return {
        name: (variants if name == "variants" else list(kind_rows.get(f"{name}.csv") or []))
        for name in _TABLE_ORDER
    }

meta_scale_errors

meta_scale_errors(
    rows: list[MetaRow], tier_scale: str | None
) -> list[str]

The one refusal that needs the module but no other table: a tier with no tier_scale (RM324).

Separate from build_meta so compile_module can ask it before mkdir while expanding after the positional fill, where its rows match the pre-flight's.

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def meta_scale_errors(rows: list[MetaRow], tier_scale: str | None) -> list[str]:
    """The one refusal that needs the module but no other table: a `tier` with no `tier_scale` (RM324).

    Separate from `build_meta` so `compile_module` can ask it before `mkdir` while expanding after the
    positional fill, where its rows match the pre-flight's.
    """
    tiered = [_line(index) for index, row in enumerate(rows) if row.tier is not None]
    if not tiered or tier_scale:
        return []
    return [
        f"{META_CSV} states `tier` on line(s) {tiered} and module_spec.yaml names no `tier_scale` — a "
        f"tier with no scale beside it is a grade nobody can read. Name the scale once in "
        f"module_spec.yaml as `tier_scale:` (free text, e.g. the rubric your README defines)."
    ]

build_meta

build_meta(
    rows: list[MetaRow],
    tables: dict[str, list[Any]],
    tier_scale: str | None,
) -> MetaBuild

Expand every member against tables, judge the rows that need the module, and resolve precedence.

Pure computation over rows already loaded, so both validate_spec and compile_module run it, on the same view, and reach the same sentences. Nothing here reads or writes a file.

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def build_meta(rows: list[MetaRow], tables: dict[str, list[Any]], tier_scale: str | None) -> MetaBuild:
    """Expand every member against `tables`, judge the rows that need the module, and resolve precedence.

    Pure computation over rows already loaded, so both `validate_spec` and `compile_module` run it, on
    the same view, and reach the same sentences. Nothing here reads or writes a file.
    """
    errors: list[str] = meta_scale_errors(rows, tier_scale)
    warnings: list[str] = []

    by_key: dict[str, dict[str | None, list[int]]] = {}
    by_gene_of: dict[str, dict[str, list[int]]] = {}
    all_members: list[list[MetaMember]] = []
    all_texts: list[list[str]] = []
    all_matches: list[list[MetaMatch]] = []
    all_withheld: list[list[str]] = []
    for index, row in enumerate(rows):
        row_members = parse_members(row.members)
        texts = split_members(row.members)
        row_matches: list[MetaMatch] = []
        withheld: list[str] = []
        for position, (authored, text) in enumerate(zip(row_members, texts, strict=True)):
            member = _as_stored(authored)
            table_rows = tables.get(member.table) or []
            locus = dict(member.conditions).get("variant_key")
            positions = (
                by_key.setdefault(member.table, _variant_key_index(table_rows)).get(locus, [])
                if locus is not None and member.table in LOCUS_TABLES
                else range(len(table_rows))
            )
            hits = [
                MetaMatch(position, member.table, k, _key(table_rows[k]))
                for k in positions
                if all(_condition_holds(table_rows[k], column, value) for column, value in member.conditions)
            ]
            if not hits:
                warnings.append(
                    CodedWarning("meta_selector_matches_nothing", _matches_nothing(index, text, member.table))
                )
            row_matches.extend(hits)
            reached = [table_rows[hit.index] for hit in hits]
            reached_tables = [member.table]
            if member.table == "allele_function" and len(row_members) == 1:
                diplotypes = tables.get("diplotypes") or []
                by_gene = by_gene_of.setdefault("diplotypes", _diplotypes_by_gene(diplotypes))
                contained = [
                    diplotypes[k] for target in reached for k in _contained(target, diplotypes, by_gene)
                ]
                reached += contained
                reached_tables += ["diplotypes"] if contained else []
            differing = _member_drug_disagrees(row, reached)
            if differing:
                warnings.append(
                    CodedWarning(
                        "meta_member_drug_disagrees",
                        _drug_disagrees(index, text, " and ".join(reached_tables), row.drug or "", differing),
                    )
                )
            if row.direction is not None and allele_bearing(member) is None:
                stated = [table_rows[hit.index].effect_allele for hit in hits]
                alleles = {allele.upper() for allele in stated if allele is not None}
                if not hits:
                    withheld.append("direction")
                elif len(alleles) != 1 or None in stated:
                    warnings.append(
                        CodedWarning(
                            "meta_member_allele_ambiguous",
                            _allele_ambiguous(index, text, [a for a in stated if a is not None]),
                        )
                    )
                    withheld.append("direction")
        all_members.append(row_members)
        all_texts.append(texts)
        all_matches.append(row_matches)
        all_withheld.append(sorted(set(withheld)))

    return MetaBuild(
        rows=rows,
        members=all_members,
        texts=all_texts,
        matches=all_matches,
        withheld=all_withheld,
        resolved=_resolve(rows, all_members, all_matches, tables, all_withheld),
        errors=errors,
        warnings=warnings,
    )

meta_frame

meta_frame(
    build: MetaBuild, module_name: str
) -> pl.DataFrame

meta.parquet: one row per authored row, in authored order — every authored cell verbatim (what reverse writes back), the stamped key, and what the compile derived: the match count, the matches as (member, table, key) structs, whether the row replaces the findings it names, and withheld.

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def meta_frame(build: MetaBuild, module_name: str) -> pl.DataFrame:
    """`meta.parquet`: one row per authored row, in authored order — every authored cell verbatim (what
    `reverse` writes back), the stamped key, and what the compile derived: the match count, the matches
    as `(member, table, key)` structs, whether the row replaces the findings it names, and `withheld`."""
    schema: dict[str, Any] = {"module": pl.Utf8}
    for name, field in MetaRow.model_fields.items():
        schema[name] = _polars_type(field.annotation)
    schema.update(
        {
            "match_count": pl.Int64,
            "matches": pl.List(_MATCH_STRUCT),
            "replaces": pl.Boolean,
            "withheld": pl.List(pl.Utf8),
        }
    )
    records = []
    for index, row in enumerate(build.rows):
        matches = [{"member": m.member, "table": m.table, "key": list(m.key)} for m in build.matches[index]]
        records.append(
            {
                "module": module_name,
                **{name: getattr(row, name) for name in MetaRow.model_fields},
                "match_count": len(matches),
                "matches": matches,
                # A matching combination replaces the single-locus findings it names (the maintainer,
                # 2026-11-04): exactly the rows in `matches`, so a consumer evaluates nothing (P1).
                "replaces": len(build.members[index]) > 1
                and any(getattr(row, c) for c in META_FINDING_COLUMNS),
                "withheld": build.withheld[index],
            }
        )
    return pl.DataFrame(records, schema=schema)

meta_resolved_frame

meta_resolved_frame(
    build: MetaBuild, module_name: str
) -> pl.DataFrame

meta_resolved.parquet: one row per (single-statement row, sex, drug) a one-member row reaches.

Sorted by member table (META_MEMBER_TABLES order), then the row's authored position in its table, then the sex scope (none, female, male), then the drug (none, then by name). tradeoff is RM327's verdict (True or null, never False). meta_rows are the 0-based meta.parquet rows that apply in the scope.

Source code in compiler/src/just_dna_compiler/compiler/meta.py
def meta_resolved_frame(build: MetaBuild, module_name: str) -> pl.DataFrame:
    """`meta_resolved.parquet`: one row per (single-statement row, sex, drug) a one-member row reaches.

    Sorted by member table (`META_MEMBER_TABLES` order), then the row's authored position in its table,
    then the sex scope (none, female, male), then the drug (none, then by name). `tradeoff` is RM327's
    verdict (`True` or null, never `False`). `meta_rows` are the 0-based `meta.parquet` rows that apply
    in the scope.
    """
    schema: dict[str, Any] = {
        "module": pl.Utf8,
        "table": pl.Utf8,
        "key": pl.List(pl.Utf8),
        "sex": pl.Utf8,
        "drug": pl.Utf8,
        **{name: _polars_type(MetaRow.model_fields[name].annotation) for name in META_FACT_COLUMNS},
        "tradeoff": pl.Boolean,
        "withheld": pl.List(pl.Utf8),
        "meta_rows": pl.List(pl.Int64),
    }
    records = [{"module": module_name, **record} for record in build.resolved]
    return pl.DataFrame(records, schema=schema)