UofA Adversarial Coverage Report

HIT HIT+ MISS WRONG CLEAN-CORRECT CLEAN-WRONG GEN-INVALID

View 1 — Catalog self-coverage (UofA pattern × subtlety)

COU-dependent rules: No rules show COU-dependent firing behavior at the 30% disparity threshold.

Pattern	low	medium	high	per-COU breakdown
COMPOUND-01	—	—	53% (96/180)	show Morrison COU1: 53% Morrison COU2: n/a Nagaraja COU1: n/a
COMPOUND-03	—	—	79% (142/180)	show Morrison COU1: 79% Morrison COU2: n/a Nagaraja COU1: n/a
W-AL-01	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-AL-02	—	—	0% (0/180)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-01	—	—	95% (170/179)	show Morrison COU1: 94% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-02	—	—	98% (176/179)	show Morrison COU1: 98% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-03	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-04	—	—	0% (0/180)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-05	—	—	100% (44/44)	show Morrison COU1: 98% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-01	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-02	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-03	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-04	—	—	100% (177/177)	show Morrison COU1: 98% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-05	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-01	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-02	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-03	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-04	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-ON-01	—	—	not measurable	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-ON-02	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-PROV-01	—	—	67% (121/180)	show Morrison COU1: 67% Morrison COU2: n/a Nagaraja COU1: n/a
W-SI-01	—	—	not measurable	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-SI-02	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a

View 2 — Literature coverage (gap_probe outcomes by taxonomy)

Source taxonomy	WRONG	INVALID	Verdict
clarissa-machinery/workflow/eliminative-argumentation	15	0	alt rule fires
clarissa-machinery/workflow/residual-risk-justification	15	0	alt rule fires
clarissa-machinery/workflow/theory-preconditions	15	0	alt rule fires
gohar/contextual/configuration	15	0	alt rule fires
gohar/contextual/environmental-factors	15	0	alt rule fires
gohar/contextual/faults-physical	15	0	alt rule fires
gohar/contextual/faults-software	15	0	alt rule fires
gohar/contextual/human-errors	15	0	alt rule fires
gohar/evidence_validity/coverage-edge-cases	15	0	alt rule fires
gohar/evidence_validity/data-drift	15	0	alt rule fires
gohar/evidence_validity/fidelity	15	0	alt rule fires
gohar/evidence_validity/inadequate-metrics	15	0	alt rule fires
gohar/evidence_validity/model-variance	15	0	alt rule fires
gohar/evidence_validity/robustness	15	0	alt rule fires
gohar/requirements/ambiguous	15	0	alt rule fires
gohar/requirements/inconsistent	15	0	alt rule fires
gohar/requirements/incorrect	15	0	alt rule fires
gohar/requirements/missing	15	0	alt rule fires
gohar/requirements/stale	15	0	alt rule fires
greenwell/sufficiency/arguing-from-ignorance	15	0	alt rule fires
greenwell/sufficiency/confusion-necessary-sufficient	15	0	alt rule fires
greenwell/sufficiency/hasty-inductive-generalization	14	1	alt rule fires

View 3 — Precision / recall summary

Metric	Value	Source battery
Catalog recall (HIT + HIT+)	69.9%	confirm_existing (n=3626)
Catalog precision (1 − FPR)	0.0%	negative_controls (n=176)
Gap-probe MISS rate	0.0%	gap_probe (n=329)

Outcome class counts

Class	Count
COV-CLEAN-WRONG	176
COV-HIT-PLUS	2626
COV-WRONG	1419
GEN-INVALID	384

View 4 — Per-rule precision (M5-B)

Decomposes the catalog's package-level precision (= 0% on M5) into per-rule contributors. NC denominator: 176 evaluable negative_control rows. Sorted by NC FPR descending.

Rule	NC FPR	NC firings	Total firings	Targeted firings	Precision when fires
`W-AL-02`	100.0%	176	382	0	0.0%
`W-ON-02`	89.8%	158	4203	180	4.3%
`W-CON-01`	19.9%	35	391	179	45.8%
`W-CON-04`	17.6%	31	313	177	56.5%
`COMPOUND-03`	15.9%	28	1331	142	10.7%
`W-AR-02`	15.9%	28	2255	182	8.1%
`COMPOUND-01`	14.8%	26	2435	96	3.9%
`W-EP-04`	2.8%	5	915	179	19.6%
`W-AR-01`	1.1%	2	172	170	98.8%
`W-AL-01`	0.0%	0	4045	178	4.4%
`W-AR-03`	0.0%	0	178	178	100.0%
`W-AR-05`	0.0%	0	3902	59	1.5%
`W-CON-02`	0.0%	0	180	180	100.0%
`W-CON-05`	0.0%	0	179	179	100.0%
`W-EP-01`	0.0%	0	186	185	99.5%
`W-EP-02`	0.0%	0	3459	178	5.1%
`W-PROV-01`	0.0%	0	121	121	100.0%

Performance characterization (v1.8 D2)

Hardware: hardware spec not configured (set UOFA_HW_SPEC env var)

Metric	Value (ms)
Mean total_eval_ms	3597
Median total_eval_ms	3532
p95 total_eval_ms	4335
Sample size	4221

Slowest 20 packages

spec_id	variant	total_eval_ms
adv-2026-p2-014-w-si-02_high_morrison-cou2	6	5487
adv-2026-p2-014-w-si-02_high_morrison-cou2	4	5470
adv-2026-p2-014-w-si-02_high_morrison-cou2	5	5428
adv-2026-p2-014-w-si-02_high_morrison-cou2	7	5326
adv-2026-p2-014-w-si-02_high_morrison-cou2	3	5167
adv-2026-p2-019-w-con-05_low_morrison-cou2	12	5154
adv-2026-p2-019-w-con-05_low_morrison-cou2	11	5115
adv-2026-p2-019-w-con-05_low_morrison-cou2	13	5070
adv-2026-p2-019-w-con-05_low_morrison-cou2	14	5032
adv-2026-p2-005-w-ep-01_medium_morrison-cou2	12	5004
adv-2026-p2-019-w-con-05_low_morrison-cou2	15	4935
adv-2026-p2-019-w-con-05_low_morrison-cou2	16	4923
adv-2026-p2-005-w-ep-01_medium_morrison-cou2	13	4904
adv-2026-p2-110-stale_low	1	4903
adv-2026-p2-107-missing_low	3	4884
adv-2026-p2-014-w-si-02_high_morrison-cou2	8	4881
adv-2026-p2-005-w-ep-01_medium_morrison-cou2	15	4876
adv-2026-p2-014-w-si-02_high_morrison-cou2	9	4862
adv-2026-p2-110-stale_low	2	4852
adv-2026-p2-005-w-ep-01_medium_morrison-cou2	14	4851

n = 4605 package rows.