just_dna_enricher.gwas¶
just_dna_enricher.gwas ¶
The NHGRI-EBI GWAS Catalog pass — published effect sizes as a derived fact table (0.6, RM90).
Fills gwas_effects.csv for the variants variants.csv names, from the Catalog's REST API v2
(RM354). What it does not do is fill weight: a consumer asked for exactly that and it is barred —
MODULE_LIFECYCLE § Stage 3 names weight/direction/effect_size in the cells no tool fills, and
every check in this tier reports rather than repairs. The effect lands in its own table beside the
authored column, and a consumer chooses between them wholesale.
The legacy API is gone. /gwas/rest/api/singleNucleotidePolymorphisms/<rsid>/associations, which
this pass read from 0.6 until 2026-11-04, answers HTTP 410 on every path with "This legacy GWAS
Catalog REST API is no longer available". A 410 now raises GwasRetired, never the plain could-not-ask
GwasError, because a source that moved and a source that is down are cleared by different things.
Everything below was established by probing v2 on 2026-11-04 (/gwas/rest/api/v2/associations?
rs_id=), not from its migration guide, and these findings shaped the pass:
- The association is fat now, and the request budget collapsed with it.
pubmed_id,accession_idandefo_traitssit on the association itself, so the legacy1 + 2Nper variant (a study and a trait link per association, measured at 382 requests for one real module) is gone. Ancestry is the one fact still behind a request —/studies/<accession>/ancestries— so the cost is one request per page plus one per distinct study, and_LinkCachekeyed on that URL now pays. - It pages, 20 per page by default. rs4149056 carries 386 associations. The pass asks for large
pages and follows
_links.nextuntil there is none, and every page is counted in the request total. A server may refuse a large page, not only cap it (RM428):size=1000answered whole on 2026-11-04 and a bare 400 six days later, when the ceiling had dropped to 500. A refused first request is asked once more with nosize, so the Catalog's own default page decides, and the run warns once naming the refused size. - The empty answer is a 200 with no
_embeddedkey at all,page.totalElements: 0— for an rsID the Catalog has never heard of as much as for a rare clinical one. The legacy API 404'd there instead, so the meaning of a 404 flipped: on the v2 search path it is a wrong path, never an answer. - The effect is one composite string,
"0.05 umol/l increase"(or"-"when the association is an odds ratio, which sits inor_per_copy_num). The legacybetaNum/betaUnit/betaDirectioncome out of it by one grammar; a string outside it is withheld whole and counted. The unit is kept verbatim between the number and the direction — one study writes U+2009 in it. snp_effect_alleleisrs4149056-C, orrs4149056-?when the study never established which allele carries the effect — the legacyriskAlleleNamegrammar, unchanged.-?becomesNone, never a guess, and the row is still written: an effect relative to an unknown allele is real evidence that cannot be used as a weight.- The unit is free text and frequently uninterpretable —
umol/lon one association anduniton many others for the same variant. It is stored verbatim including the useless values, because "these betas are on unknown and possibly different scales" is the fact a consumer must have. - v2 publishes no standard error.
standard_errorstays null on every row this pass writes. - The p-value has its curated pair back.
p_valueis0.0past float64's range, as before, butpvalue_mantissa/pvalue_exponentbeside it still say3e-493; the verbatim string is built from the pair, and the queryable number is read from that same string.
The published budget is generous, and the gate is not it. The v2 docs page
(https://www.ebi.ac.uk/gwas/rest/api/v2/docs, "Throttling", read 2026-11-11) says "Users of the API are
limited to 15 queries per second ... If the limit is exceeded, the API will slow down subsequent calls."
An excess slows rather than refuses, unlike gnomad.py, whose 10-requests-per-60-seconds is load-bearing.
The gate here (one request a second) stays a conservative default well under that budget, not a
transcription of it, so nobody "corrects" it up to 15 per second against a server that never refused.
The page states no page-size ceiling; RM428's 500 is measured only.
GwasError ¶
Bases: RuntimeError
A GWAS Catalog fetch or parse failed in a way the caller must see.
Every transport and parse failure is translated into this before it leaves the module. A client
that leaks httpx.HTTPError has no contract — the caller would have to import the transport
library to catch it, and a swap of transport would silently become a breaking change.
GwasNotFound ¶
Bases: GwasError
The Catalog answered, and it has no record of this variant at all (HTTP 404).
A subclass rather than a flag, because this is an answer and not a failure. Found by running
the pass against a real module: the Catalog holds only variants with a published genome-wide
association, so rs111033563 — a rare clinical HFE variant — 404s rather than returning an empty
list. The first version of this pass treated that as an outage and died on the first rare variant
in the module, which is every clinically-authored module.
It stays inside GwasError so a caller that does not care about the distinction still catches
one type. Since v2 (RM354) only follow treats it as an answer — one study's ancestry
withheld. The v2 search answers a variant it holds nothing for with a 200 and zero elements, and
its only measured 404 was a path that does not exist, so association_pages lets this propagate:
reading it as "no associations" would write a confident negative about a variant nobody could ask
about.
GwasRetired ¶
Bases: GwasError, EndpointRetired
The Catalog answered HTTP 410 Gone: it retired the endpoint this client asked (RM354).
A fact about the source, not an outage. The legacy REST API began answering 410 on every path
with "This legacy GWAS Catalog REST API is no longer available", and the generic status arm
turned that into a plain GwasError — the same could-not-ask an outage raises — so a source
that had moved read as one that was down, and nothing said to upgrade. The message carries the
URL and the source's own text, which names where it went.
Not retried: a 410 is an HTTPStatusError, which is outside the retry predicate by construction,
and asking again changes nothing. A subclass of GwasError so every existing except GwasError
still catches it; nothing in the tier catches GwasNotFound before GwasError on a path that
could see a 410, and follow's except GwasNotFound does not match it, so it propagates.
GwasResult
dataclass
¶
GwasResult(
rows: list[GwasEffectRow] = list(),
covered: list[str] = list(),
missing: list[str] = list(),
requests_made: int = 0,
requests_saved: int = 0,
p_value_underflows: int = 0,
unparsed_effects: int = 0,
unusable: int = 0,
skipped_offline: bool = False,
)
What one pass did, including what it spent.
GwasCatalogClient
dataclass
¶
GwasCatalogClient(
endpoint: str = DEFAULT_GWAS_ENDPOINT,
stats_url: str = DEFAULT_GWAS_STATS_URL,
timeout: float = 30.0,
gate: PacingGate = (
lambda: PacingGate(DEFAULT_REQUEST_INTERVAL)
)(),
size_refusals: int = 0,
_client: Client | None = None,
)
Thin REST client for the Catalog, paced and retried.
endpoint and gate are injectable so a test can drive the real parsing code against recorded
payloads without a network or a real sleep — the same shape EnsemblResolver and the gnomAD
client use.
release ¶
The release date the Catalog states, or None when it could not be read or states none (RM373).
Asked once per run. A failure is the unknown arm, logged, never an error: the associations are the run's work, and a release that could not be read is withheld from their label, never replaced by the day the run happened.
Source code in enricher/src/just_dna_enricher/gwas.py
association_pages ¶
Every page of the Catalog's associations for one rsID, one list per request made.
Pages rather than a flat list so the pass can count what it spent (@dont-discard-computed).
Three outcomes, not two — the S20 shape. Associations are an answer; a single empty page
is also an answer (the Catalog was reached and holds nothing for this variant, which the
caller records as not_found); and a raised GwasError means it could not be asked, which is
unchecked rather than empty. Fusing the last two would turn a failed request into a definite
negative — which is why a 404 here raises (see GwasNotFound).
Source code in enricher/src/just_dna_enricher/gwas.py
study_ancestries_url ¶
Where v2 keeps one study's ancestries. Built from the accession: v2 associations carry no
_links.study, only self, loci and snp.
Source code in enricher/src/just_dna_enricher/gwas.py
follow ¶
A linked sub-resource, or {} when the Catalog has none.
A 404 here withholds the ancestry for one association rather than sinking the pass: the
association itself is still a real published effect, and dropping it because its study record
moved would lose evidence over metadata. A 410 is not caught: GwasRetired is the source
moving, and swallowing it here would null every row's ancestry and say nothing.
Source code in enricher/src/just_dna_enricher/gwas.py
enrich_gwas ¶
enrich_gwas(
spec_dir: Path,
*,
mode: str = "best_effort",
offline: bool = False,
write: bool = True,
client: GwasCatalogClient | None = None,
dataset: str | None = None,
declared_use: str = "unstated",
study_facts: bool = True,
) -> GwasResult
Record the Catalog's published effect sizes for this module's variants.
Existing rows are authoritative and merged, never clobbered — the standing rule for every pass, with the standing consequence: to regenerate after a machinery change, delete the file first.
offline makes the pass a no-op with a warning rather than a failure: the Catalog publishes a
bulk download, but this pass reads the REST API, and there is no snapshot for it to fall back on.
An injected client still wins, because handing over a transport you already hold is not egress.
A variant the Catalog holds nothing for gets a not_found row, unlike
enrich_gene_validity's silence, and the difference is real rather than stylistic: a curating
body's silence means nobody has assessed the gene, whereas the Catalog's empty answer means no
genome-wide association has been published for this variant — which is a fact about it, and one
a consumer weighing an authored weight against the literature wants to see.
mode is the severity ladder the CLI's --strict sets, and it is deliberately not about
missing: see the block above the raise at the end of this function for why the Catalog's empty
answer is a fact rather than a shortfall, and what strict reads instead.
Source code in enricher/src/just_dna_enricher/gwas.py
675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 | |
gwas_source_row ¶
The licence row this pass writes. Exposed so a caller can record the terms without fetching.