just_dna_compiler.compiler.parquets¶
just_dna_compiler.compiler.parquets ¶
Building the SNP-core and frequency parquets: weights, annotations, studies, frequencies.
resolution_loci ¶
resolution_loci(
key: str | None,
resolution: dict[str, list[ResolutionRow]],
genome_build: str,
*,
table_read: bool = False,
) -> list[dict[str, Any]] | None
The loci resolution.csv files under key on this build, in locus_index order, or None.
Why a parquet carries them. reverse_module rebuilds resolution.csv from parquets, so a fact
no parquet holds does not come back, resolution_signature moves, and whatever the compile said
about it goes silent on the second lap. Two kinds of row are keyed into the table without holding
its answer in their own columns: a study's LD proxy, which is usually not a module variant (RM331),
and a variant row the fill left alone (RM364) — one whose authored coordinate the table contradicts,
one whose key names several loci none of which was taken, a locus dropped as unable to host the
genotype, and every row of a module on a build the fill skips. Rebuilding those from the row's own
cells wrote the author's coordinate back as a resolution fact.
Positioned loci only: a not_found row carries no fact for reverse to restore.
An empty list says the table filed nothing under this key (RM451). None already means "the
row's own cells answer" (withhold_reproduced_loci, or a parquet compiled before the column, or a
module that read no table), and reverse rebuilds from the cells there. A coordinate row a module's
table never filed (authored after the last enrich) was read the same way, so reverse invented a
table row for it and the second lap read a row the first did not. A module that read a table
therefore says [] for a key under which it holds no positioned locus on this build, and reverse
writes nothing for it. table_read is
whether a table was read at all (InjectedResolution.consulted), never whether it has rows: reverse
hands an empty table back as a header-only file, and the second lap must say [] where the first did.
Source code in compiler/src/just_dna_compiler/compiler/parquets.py
cell_fact ¶
What reverse's cell rebuild writes for one row, over every CELL_FACT_FIELDS column, in order.
cells holds the row's value for each of those columns, a resolution_loci locus holds them as
stored, and the two compare equal exactly when the cell rebuild would write the locus back. Indexed:
every caller states each column, the positional one by row.get because _resolution_record reads
a kind's missing column as empty too. The defect this replaced (RM389) was a column not compared at
all; one compared as empty against a locus holding a value keeps that locus, the safe direction.
locus_index absent is 0, the ordinal reverse gives a row that names none.
Source code in compiler/src/just_dna_compiler/compiler/parquets.py
withhold_reproduced_loci ¶
withhold_reproduced_loci(
cells: list[list[dict[str, Any]] | None],
keys: list[str | None],
facts: list[tuple[Any, ...] | None],
) -> list[list[dict[str, Any]] | None]
cells with a key's loci nulled wherever its rows' own cells already rebuild them exactly (RM364).
facts[i] is what reverse's cell rebuild writes for row i (None where it writes nothing, a row
with no coordinate). Where the set of those facts over a key equals the facts of the loci the table
files under it, the cells answer and the column stays null, so a module the fill resolved cleanly
carries nothing new, and its artifact.digest matches the deprecated ensembl_cache path's, which
reads no table. Everywhere else, which is every row the fill left alone, the loci are kept.
Since RM380 a locus holding provenance is kept too — authority, a placement witness,
left_aligned, source_release, an ambiguous status, RM375's verdict — because no cell holds those
and withholding the locus would drop them on the round trip. So the "carries nothing new" case is
now a table with no provenance (a hand-filled or reversed-from-old table, or the deprecated path's),
and an enriched module carries its loci on every keyed row: a parquet column, priced near zero (P9).
Source code in compiler/src/just_dna_compiler/compiler/parquets.py
with_resolution_loci ¶
with_resolution_loci(
table: DataFrame,
resolution: dict[str, list[ResolutionRow]],
genome_build: str,
*,
table_read: bool = False,
) -> pl.DataFrame
A positional table's parquet with resolution_loci: the loci filed under each row's key (RM364).
Keyed on the row's stored variant_key, the key the positional fill looked the row up by and the
key reverse files it under. Kept wherever the row's own cells would not rebuild them
(withhold_reproduced_loci): a row the fill left alone still carries what the table said about it.
An empty resolution (no table, or --no-resolve) leaves the column null.