just_dna_enricher.locations¶
just_dna_enricher.locations ¶
Ensembl reference cache resolution — mirrors just-dna-lite's on-disk layout so a marketplace or
standalone compile can reuse an existing just-dna-lite deployment's cache (disk economy, no
re-download), pointed via .env.
Layout (identical to just-dna-pipelines)::
<base>/ensembl_variations/data/*.parquet
<base>/ensembl_variations/ensembl_variations.duckdb # optional prebuilt view
where <base> is $JUST_DNA_PIPELINES_CACHE_DIR (the same var just-dna-lite uses), or the
platformdirs user cache for "just-dna-pipelines". $JUST_DNA_ENSEMBL_CACHE (a .duckdb
file or a directory) overrides everything for explicit pointing.
This module never downloads: if no cache is present, resolution returns None and the
resolver skips with a warning. Provisioning the reference is the deployment's job.
SnapshotManifestFile ¶
Bases: BaseModel
One file manifest.json attests: its path under the snapshot root, its size and its sha256.
Unstated is None, and a None size or digest attests nothing. Keys this reader does not type are
dropped rather than carried (extra="ignore"), so nothing downstream can read one untyped.
SnapshotManifest ¶
Bases: BaseModel
prepare-annotations' manifest.json, the record its Ensembl release carries instead of release.json.
Typed for the four things a reader here acts on: which release (release_id), when it was cut,
the layout version, and the files it attests. The rest of the file (the VEP header, the column
groups, per-chromosome counts) is the builder's own and is not read here.
entry ¶
The attestation for relative (data/<name>), or None when the manifest names no such file.
Source code in enricher/src/just_dna_enricher/locations.py
ManifestCoverage
dataclass
¶
How a cache's data/ meets its own manifest.json, in both directions (RM372).
unattested: payload files present that the manifest does not name, or names at another size,
so another release's. absent: files the manifest attests under data/ that are not there, so
the snapshot is incomplete. Either one withholds the whole cache; neither is a guess about the
other. Files outside data/ (prepare-annotations' groups/*) are the manifest's and not pulled,
so they are not counted as absent.
whole
property
¶
Every payload file is attested and every attested data/ file is present.
repro_out ¶
The default --out for a builder, under data/repro/<name>/.
Derived rather than restated, because a rule spelled out once per command is a rule that holds
until somebody adds the tenth command. An AST guard over the CLI asserts every --out default
resolves under data/, so a new builder that writes its own literal fails the suite rather than
the operator's working tree.
Source code in enricher/src/just_dna_enricher/locations.py
parquet_footer_ok ¶
A complete parquet begins and ends with the PAR1 magic. A truncated/interrupted download —
the common cache-corruption mode — is missing its footer, which is what makes DuckDB blow up with
"No magic bytes found at end of file". Reads only 4 bytes from each end.
Here rather than in download because a resolver judges a recorded file by it too (RM339), and
this module never imports the network tier.
Source code in enricher/src/just_dna_enricher/locations.py
snapshot_record_missing ¶
The files cache_dir's pull record names that are absent or truncated, or None with no record.
Three answers, and a caller branches on all three (RM339). []: the pull that wrote the record
finished, so the cache is complete. A non-empty list: a pull was planned and did not finish, so the
cache is not a snapshot yet, and these are what it lacks. None: no record, so nothing here can
say. That is a cache built locally, or one pulled before the record existed; the resolvers treat
it as they always did, and the provisioner compares it with the remote listing once.
A record that is present and unreadable is None with a warning. The provisioner rewrites it on
its next pull, and refusing a whole cache over its bookkeeping would cost a reader its snapshot.
Source code in enricher/src/just_dna_enricher/locations.py
pull_record_repo ¶
The repo cache_dir's bytes were pulled from, by its pull record, or None when it says none.
During an unfinished repo switch that is the repo being replaced (PULL_RECORD_REPLACING_KEY),
not the one the record names: some files are still the old repo's, and nothing but the record can
say so when no manifest.json attests them. None covers no record, an unreadable one, and one
written before the repo was recorded: none of them states another repo, so none is taken for one.
Since the Ensembl lane moved to a new repo under the same file names, this is what tells a cache
of the old release from one of the new where no manifest is present.
Source code in enricher/src/just_dna_enricher/locations.py
pulled_from ¶
The repo cache_dir was pulled from when its pull record names one other than repo, else None.
The one answer to "is this cache another repo's snapshot", for the provisioner and for cache pull.
A cache that names no repo is not another repo's: only a record naming one is evidence.
Source code in enricher/src/just_dna_enricher/locations.py
read_snapshot_manifest ¶
The manifest at path (a file), or None when it is absent or cannot be read whole.
A caller that has to tell the two apart checks path.is_file(): present and unreadable is a
refusal (manifest_coverage attests nothing under it), absent is the unknown arm.
Source code in enricher/src/just_dna_enricher/locations.py
manifest_attests ¶
Whether local is the file manifest states for relative, by size and then by sha256.
The full check, for the provisioner, which already pays for the bytes. The size comparison comes
first so a file of another release costs a stat, not a hash.
Source code in enricher/src/just_dna_enricher/locations.py
manifest_coverage ¶
How cache_dir's data/*.parquet meets its manifest.json, or None with no manifest.
Three answers (RM372). None: no manifest, so nothing here can say which release each file is
(the old repo published none) and a reader treats the cache as it always did. whole: every
payload file is one the manifest names at the size it states, and every file it attests under
data/ is present. Otherwise the cache is withheld whole, never merged and never read in
part: old chrMT beside new chr21 answered rsIDs and nothing complained, and 12 of 25 attested
files with no pull record were served as the release. A manifest present and unreadable attests
nothing, so every payload file is unattested and none is counted absent.
Read time compares the manifest's bytes, never its sha256. Hashing 14.5 GB on every enrich
is not a read; a size is a stat, and every core file of the two releases differs in size. The
sha256 is compared where the bytes are already being paid for, by the provisioner
(manifest_attests), which never keeps a file the manifest does not attest by digest.
Source code in enricher/src/just_dna_enricher/locations.py
ensembl_release_label ¶
The release an Ensembl snapshot states: release.json's dataset, else manifest.json's release_id.
The lane's own label (CacheLane.release_label, @the-reporter-cannot-compose-a-lanes-label),
because the split repo states its release only in its manifest. None when the snapshot states
neither, which the old repo's never did: an unknown, withheld rather than guessed. A .duckdb
pointed at directly states nothing either.
Source code in enricher/src/just_dna_enricher/locations.py
load_env ¶
Load the nearest .env (walking up from CWD) into os.environ, so cache paths can be set there.
Returns the loaded path, or None.
Every variable in the file is exported, so only the cache resolvers call this. A credential is
read with env_value, which exports nothing (RM301, S124).
Source code in enricher/src/just_dna_enricher/locations.py
env_value ¶
One variable, from the process environment or else the nearest .env, without exporting the file.
This is how a credential is read where it is used (@credential-where-read). The precedence is
load_env's: override=False keeps a variable that is present, so an exported value wins over the
file, and an exported empty string stays empty. The difference is that nothing is written into
os.environ. RM301 (S124): every client used to call load_env(), which copied the whole .env
into the host's environment. A host that reports which layer each of its own settings came from
then saw every file value as an exported shell variable.
The file is found per call, walking up from the CWD, like load_env. None means neither source
names the variable. An empty string means one of them sets it empty.
Source code in enricher/src/just_dna_enricher/locations.py
missing_credential_reason ¶
Why $var is unusable — absent and exported empty are two states, not one.
load_env uses override=False, so a variable that is present is kept whatever the .env
says — and an empty string is present. That makes export FOO= strictly stronger than deleting
the variable, which is the opposite of what anyone expects and is the same edge the tier's own
tests exploit deliberately (a test neutralizes a credential with "" precisely because delenv
would let the developer's .env refill it).
It bites for real: a shell that ran a snippet whose placeholder was edited out — export
PHARMVAR_API_KEY= — reports no key for the rest of the session on a machine whose .env holds
a working one, and nothing in the message says why. So the two readings are named separately and
the empty one carries its own remedy, because unset and "go and get a key" are different
actions (@rsid-absent-two-readings is the same rule about a different absence).
Source code in enricher/src/just_dna_enricher/locations.py
default_ensembl_cache_dir ¶
The <base>/ensembl_variations directory, matching just-dna-lite's convention.
resolve_ensembl_reference ¶
resolve_ensembl_reference(
ensembl_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a usable Ensembl reference without downloading.
Precedence: explicit ensembl_cache → $JUST_DNA_ENSEMBL_CACHE → the just-dna-lite layout
under $JUST_DNA_PIPELINES_CACHE_DIR / platformdirs. Prefers a prebuilt
ensembl_variations.duckdb; otherwise the directory of parquet files. Returns the resolved
path (a .duckdb file or a directory), or None if nothing is present.
Source code in enricher/src/just_dna_enricher/locations.py
default_clinvar_cache_dir ¶
The <base>/clinvar directory (same base as the Ensembl cache).
default_constraint_cache_dir ¶
The <base>/gnomad_constraint directory (same base as the other two caches).
Source code in enricher/src/just_dna_enricher/locations.py
default_clinpgx_cache_dir ¶
The <base>/clinpgx directory — the ClinPGx clinical-annotation snapshot.
default_cpic_cache_dir ¶
The <base>/cpic directory — the CPIC allele/diplotype/recommendation snapshot.
default_pharmvar_cache_dir ¶
The <base>/pharmvar directory — operator-built only (see PHARMVAR_SUBDIR).
default_pubmind_cache_dir ¶
The <base>/pubmind directory — operator-built only (see PUBMIND_SUBDIR).
resolve_constraint_reference ¶
resolve_constraint_reference(
constraint_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a usable gnomAD constraint snapshot without downloading.
Explicit argument → $JUST_DNA_GNOMAD_CONSTRAINT_CACHE → $JUST_DNA_PIPELINES_CACHE_DIR
/platformdirs gnomad_constraint/. Parquet only (like ClinVar, there is no prebuilt
.duckdb), and a bare .parquet may be pointed at directly since the snapshot is one file.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_clinvar_reference ¶
resolve_clinvar_reference(
clinvar_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a usable ClinVar reference without downloading.
Mirrors resolve_ensembl_reference's precedence: explicit clinvar_cache →
$JUST_DNA_CLINVAR_CACHE → $JUST_DNA_PIPELINES_CACHE_DIR/platformdirs clinvar/. The
ClinVar snapshot ships as parquet only (no prebuilt .duckdb); a directory is returned when it
holds data/*.parquet (or bare *.parquet), else None. Never downloads — provisioning is
the enricher's download.ensure_clinvar_snapshot or the deployment's job.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_clinpgx_reference ¶
resolve_clinpgx_reference(
clinpgx_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built ClinPGx snapshot without downloading ($JUST_DNA_CLINPGX_CACHE).
The builder shipped a release before this existed, so the snapshot was reachable only by handing
enrich_clinpgx an explicit --snapshot; with no path the pass skipped itself and said so, which
on a hosted deployment is the check simply not running. Provisioning is
download.ensure_clinpgx_snapshot.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_cpic_reference ¶
resolve_cpic_reference(
cpic_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built CPIC snapshot without downloading ($JUST_DNA_CPIC_CACHE).
Source code in enricher/src/just_dna_enricher/locations.py
resolve_pharmvar_reference ¶
resolve_pharmvar_reference(
pharmvar_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate an operator-built PharmVar snapshot ($JUST_DNA_PHARMVAR_CACHE).
There is deliberately no download.ensure_pharmvar_snapshot to pair with this — see
PHARMVAR_SUBDIR. A deployment builds its own with its own key, or this returns None and the
PharmVar leg degrades exactly as it does when no key is configured.
Source code in enricher/src/just_dna_enricher/locations.py
default_civic_cache_dir ¶
The <base>/civic directory — operator-built for now (see CIVIC_SUBDIR).
resolve_civic_reference ¶
resolve_civic_reference(
civic_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate an operator-built CIViC snapshot ($JUST_DNA_CIVIC_CACHE).
None when there is none, and the drafter reads that as nobody-asked rather than as an empty
source (@unreachable-not-absent). Build one with civic build --release <date>.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_pubmind_reference ¶
resolve_pubmind_reference(
pubmind_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate an operator-built PubMind snapshot ($JUST_DNA_PUBMIND_CACHE).
There is deliberately no download.ensure_pubmind_snapshot to pair with this — see
PUBMIND_SUBDIR. A deployment builds its own with pubmind build, or this returns None and the
PubMind leg reads unchecked rather than absent: nobody asked is a third state beside
asked-and-failed and asked-and-absent (@unreachable-not-absent).
Source code in enricher/src/just_dna_enricher/locations.py
default_mane_cache_dir ¶
The <base>/mane directory — operator-built only (see MANE_SUBDIR).
default_opentargets_cache_dir ¶
The <base>/opentargets directory — see OPENTARGETS_SUBDIR.
resolve_opentargets_reference ¶
resolve_opentargets_reference(
opentargets_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built Open Targets snapshot ($JUST_DNA_OPENTARGETS_CACHE), without downloading.
None is nobody-asked, never "Open Targets holds nothing for this variant": the pass that reads it
says it skipped rather than writing an empty table (@unreachable-not-absent). A bare .parquet
is not accepted: the snapshot is several tables read by name.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_mane_reference ¶
resolve_mane_reference(
mane_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate an operator-built MANE snapshot ($JUST_DNA_MANE_CACHE).
There is deliberately no download.ensure_mane_snapshot to pair with this: nothing publishes a
MANE snapshot, and NCBI's policy neither grants nor withholds permission to (see MANE_SUBDIR).
None when there is none, and a reader must treat that as nobody-asked rather than as a gene
MANE has no transcript for — the snapshot's own negative roster is what answers the second
question, and it can only answer it once the snapshot exists (@unreachable-not-absent).
A bare .parquet is not accepted here, unlike the constraint snapshot: this cache is three
tables read by filename, so a single file pointed at directly would be a snapshot missing the
currency check and the negative roster.
Source code in enricher/src/just_dna_enricher/locations.py
default_acmg_cache_dir ¶
The <base>/acmg_sf directory — operator-built only (see ACMG_SUBDIR).
resolve_acmg_reference ¶
resolve_acmg_reference(
acmg_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built ACMG SF snapshot ($JUST_DNA_ACMG_CACHE), without downloading.
None means nobody provisioned one, and check-acmg reads that as leave to fall back to
scraping NCBI's page — which serves v3.2 while ACMG published v3.3 in June 2025, so a
correctly authored v3.3 row is reported as wrong by the fallback. That is the whole reason this
resolver exists: the snapshot was buildable and unreachable unless a caller passed --sf-list by
hand, so the accurate list was the one path a deployment never took.
Source code in enricher/src/just_dna_enricher/locations.py
default_strchive_cache_dir ¶
The <base>/strchive directory — see STRCHIVE_SUBDIR.
resolve_strchive_reference ¶
resolve_strchive_reference(
strchive_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built STRchive catalogue ($JUST_DNA_STRCHIVE_CACHE), without downloading.
None is nobody-asked rather than "STRchive lists no such locus", and check-repeat-bands says
so in those words (@unreachable-not-absent). A directory is preferred over a bare
STRchive-loci.json because only the directory carries release.json, and without it the check
cannot name the release it compared against.
Source code in enricher/src/just_dna_enricher/locations.py
default_mitomap_cache_dir ¶
The <base>/mitomap directory — see MITOMAP_SUBDIR.
resolve_mitomap_reference ¶
resolve_mitomap_reference(
mitomap_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built MITOMAP snapshot ($JUST_DNA_MITOMAP_CACHE), without downloading.
None is nobody-asked, not "MITOMAP publishes no such variant". It is also the parent absence the
derived miss lane reports as could not run rather than as an empty increment: a miss set derived
with one parent missing would be a claim about a comparison that never happened.
Source code in enricher/src/just_dna_enricher/locations.py
default_alphagenome_avi_cache_dir ¶
The <base>/alphagenome_avi directory — see ALPHAGENOME_AVI_SUBDIR.
Source code in enricher/src/just_dna_enricher/locations.py
resolve_alphagenome_avi_reference ¶
resolve_alphagenome_avi_reference(
alphagenome_avi_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built AVI snapshot ($JUST_DNA_ALPHAGENOME_AVI_CACHE), without downloading.
None here is nobody-asked in its strongest form: this lane has no acquire stage at all, so an
absent snapshot means the operator has not run alphagenome build against a file they hold — not
that a fetch failed, and not that AlphaGenome scores nothing at that position
(@unreachable-not-absent).
Source code in enricher/src/just_dna_enricher/locations.py
default_mitomap_miss_cache_dir ¶
The <base>/mitomap_miss directory — see MITOMAP_MISS_SUBDIR.
resolve_mitomap_miss_reference ¶
resolve_mitomap_miss_reference(
mitomap_miss_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built MITOMAP-miss snapshot ($JUST_DNA_MITOMAP_MISS_CACHE), without downloading.
A resolver for a derived snapshot answers a narrower question than the others: it says the join
was run and its result is on disk, never that the result is still current. Currency is
mitomap_miss_build.stale_parents, which re-reads both parents' release.json and names the one
that moved — a snapshot present and stale is a different answer from absent, and the drafter says
which (@currency-asks-the-source-not-the-cache).
Source code in enricher/src/just_dna_enricher/locations.py
default_clinvar_gkm_cache_dir ¶
The <base>/clinvar_gkm directory — see CLINVAR_GKM_SUBDIR.
resolve_clinvar_gkm_reference ¶
resolve_clinvar_gkm_reference(
clinvar_gkm_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built ClinVar-GKM indel snapshot ($JUST_DNA_CLINVAR_GKM_CACHE), without downloading.
Present says the slice is on disk, never that it is current: the source overwrites its one Parquet
set every week, so currency is clinvar_gkm_build.lane_currency, which asks the source.
Source code in enricher/src/just_dna_enricher/locations.py
default_drug_labels_cache_dir ¶
The <base>/drug_labels directory — see DRUG_LABELS_SUBDIR.
resolve_drug_labels_reference ¶
resolve_drug_labels_reference(
drug_labels_cache: Path | None = None,
*,
load_dotenv_file: bool = True,
) -> Path | None
Locate a built regulator drug-label snapshot ($JUST_DNA_DRUG_LABELS_CACHE).
A parquet snapshot like ClinPGx's, and a separate cache from it on purpose: the two archives
are dated from their own CREATED_*.txt and do not refresh together (DRUG_LABELS_SUBDIR).
Provisioning is download.ensure_drug_labels_snapshot.