just_dna_format.sources¶
just_dna_format.sources ¶
The data-source licensing table (0.5).
sources.csv is the fourth derived-fact sidecar, and the second not keyed on a variant. A row
records what a source is, and on what terms it was used — one row per (source, layer). Filled by
just-dna-enricher (the only tier that fetches, and therefore the only tier that knows), consumed
and hashed by the compiler, never fetched by it.
Why this exists. The pharmacogenomics sources are all copyleft, and they differ in ways that decide what a module may be used for: ClinPGx, CPIC and PharmVar are each CC BY-SA 4.0 plus a contractual clause barring sale, while Ensembl and dbSNP are unrestricted. That obligation attaches when the derivative work is made — at compile — and nothing in the artifact recorded it. A consumer holding a compiled module could not tell whether it carried a ShareAlike obligation, and the marketplace could not filter on one.
Why it is data and not a table in the compiler. A hardcoded {"clinpgx": "CC-BY-SA-4.0", …} map
in just-dna-compiler would give that tier a source convention, which Principle 2 removed from it in
0.5, and would be an un-injected reference — the same category as holding a ClinVar snapshot. It
would also be permanently wrong: api.pharmgkb.org was retired mid-0.5 and CPIC's licence URL moved
in the same period. So the licence travels as data, read by the enricher from the bytes it actually
downloaded (ClinPGx ships a LICENSE.txt inside each archive) and pinned by license_sha256, which
makes the recorded terms provably contemporaneous with the recorded data.
Why a fact table rather than a column on ResolutionRow. _write_resolution_csv rebuilds that
table from weights.parquet, which by design carries no provenance at all, with a hardcoded field
list and source reset to "reversed". A licence column there would be wiped on every
compile → reverse → compile cycle and could never be recovered — the same structural reason
rsid_alternates is unrecoverable. As its own fact table it round-trips for free, because
_write_table_csv derives its columns from model_fields.
Tri-state booleans are load-bearing. share_alike and commercial_use are Optional[bool], and
None means unknown, never false. A source whose terms could not be established has not been
shown to permit anything, and rendering that as "does not forbid" is the single most dangerous
simplification available here. LiteratureRow.quotes_found carries the same null-is-not-zero
contract for the same reason.
SourceRow ¶
Bases: BaseModel
One data source, at one layer of the module, with the terms it was used under.
Standalone (not an AuthoredModel) for the same reason ResolutionRow/FrequencyRow/
GeneMetricsRow/LiteratureRow are — a machine-produced reference fact rather than an authored
annotation — with extra="forbid" so a typo'd column is caught rather than silently dropped.
But it is the one of the five a human starts from a template, and that is why it carries the
placeholder guard below (RM76). S21 put it in DRAFTABLE precisely because it is "the only
fact sidecar a human writes" and the only table the compile licence gate reads; the standalone
decision above predates that and nobody reconciled the two. The consequence was measured on
reference_examples/hfe_hemochromatosis with source=<<REPLACE>>: the module compiles green
under --strict and manifest.sources publishes "sources": ["<<REPLACE>>"] — inside the
block its own signature is computed over. A signed module's attribution ledger naming a
template placeholder as the source it accounts for is the one thing this table exists to prevent.
A vocabulary column caught a stub by accident (layer refuses the token as a non-member); a
free-text one did not, and source is free text by design. Reaching the guard rather than the
base is deliberate — ModuleSpecConfig is the precedent, standalone for its own reasons and
guarded all the same — so the classification above stays true and the other four sidecars stay
out, since no template is ever generated for them.
taints_commercial_use ¶
Whether this row alone makes the module non-sellable.
Two conditions, both required. The source must actually forbid sale (commercial_use is False —
an unknown does not taint, it warns, because "we could not read the terms" is not a finding
that they forbid anything). And it must sit at the annotation layer, where expression is
embedded; a source consulted only for a coordinate contributed a fact that Ensembl reports
identically, and marking that as viral is the false-positive this predicate exists to prevent.
Shared so the compiler's gate and the manifest summary cannot drift apart.
Source code in schema/src/just_dna_format/sources.py
taints_redistribution ¶
Whether this row alone makes the module non-redistributable.
Same two conditions and the same reasoning as taints_commercial_use, on the third axis: only an
annotation-layer source can taint (a coordinate looked up from a restricted service is still
just a coordinate), and an unknown does not taint.
Recorded and summarized, and deliberately NOT gated in any of these packages. Settled in 0.6,
not still open (RM27). The question was whether a redistribution bar should resolve against
declared_use, and the answer is that it cannot: a distribution right is not a use, so the
three-state unstated|non_commercial|commercial axis has nothing to say about it. Gating on the
act is right, and the act is a publish — which happens downstream, in a registry, not in a
compile. So the verdict is stamped into manifest.sources.redistribution and the enforcement is
asked for by name in docs/SCHEMAS.md, addressed to the registry, for 0.6 integration.
What is rejected and stays rejected is a second author declaration beside declared_use: it is
symmetric and tempting, and it asks an author at build time about something they may not know
until later.