UofA Adversarial Coverage Report

HIT HIT+ MISS WRONG CLEAN-CORRECT CLEAN-WRONG GEN-INVALID

View 1 — Catalog self-coverage (UofA pattern × subtlety)

COU-dependent rules: No rules show COU-dependent firing behavior at the 30% disparity threshold.

Pattern	low	medium	high	per-COU breakdown
COMPOUND-01	—	—	61% (110/180)	show Morrison COU1: 61% Morrison COU2: n/a Nagaraja COU1: n/a
COMPOUND-03	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-AL-01	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-AL-02	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-01	—	—	95% (170/179)	show Morrison COU1: 94% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-02	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-03	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-04	—	—	0% (0/180)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-AR-05	—	—	100% (44/44)	show Morrison COU1: 98% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-01	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-02	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-03	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-04	—	—	100% (177/177)	show Morrison COU1: 98% Morrison COU2: n/a Nagaraja COU1: n/a
W-CON-05	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-01	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-02	—	—	100% (178/178)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-03	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-EP-04	—	—	100% (179/179)	show Morrison COU1: 99% Morrison COU2: n/a Nagaraja COU1: n/a
W-ON-01	—	—	not measurable	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-ON-02	—	—	100% (180/180)	show Morrison COU1: 100% Morrison COU2: n/a Nagaraja COU1: n/a
W-PROV-01	—	—	67% (121/180)	show Morrison COU1: 67% Morrison COU2: n/a Nagaraja COU1: n/a
W-SI-01	—	—	not measurable	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a
W-SI-02	—	—	0% (0/179)	show Morrison COU1: 0% Morrison COU2: n/a Nagaraja COU1: n/a

View 2 — Literature coverage (gap_probe outcomes by taxonomy)

Source taxonomy	WRONG	INVALID	Verdict
clarissa-machinery/workflow/eliminative-argumentation	15	0	alt rule fires
clarissa-machinery/workflow/residual-risk-justification	15	0	alt rule fires
clarissa-machinery/workflow/theory-preconditions	15	0	alt rule fires
gohar/contextual/configuration	15	0	alt rule fires
gohar/contextual/environmental-factors	15	0	alt rule fires
gohar/contextual/faults-physical	15	0	alt rule fires
gohar/contextual/faults-software	15	0	alt rule fires
gohar/contextual/human-errors	15	0	alt rule fires
gohar/evidence_validity/coverage-edge-cases	15	0	alt rule fires
gohar/evidence_validity/data-drift	15	0	alt rule fires
gohar/evidence_validity/fidelity	15	0	alt rule fires
gohar/evidence_validity/inadequate-metrics	15	0	alt rule fires
gohar/evidence_validity/model-variance	15	0	alt rule fires
gohar/evidence_validity/robustness	15	0	alt rule fires
gohar/requirements/ambiguous	15	0	alt rule fires
gohar/requirements/inconsistent	15	0	alt rule fires
gohar/requirements/incorrect	15	0	alt rule fires
gohar/requirements/missing	15	0	alt rule fires
gohar/requirements/stale	15	0	alt rule fires
greenwell/sufficiency/arguing-from-ignorance	15	0	alt rule fires
greenwell/sufficiency/confusion-necessary-sufficient	15	0	alt rule fires
greenwell/sufficiency/hasty-inductive-generalization	14	1	alt rule fires

View 3 — Precision / recall summary

Metric	Value	Source battery
Catalog recall (HIT + HIT+)	76.4%	confirm_existing (n=3626)
Catalog precision (1 − FPR)	4.5%	negative_controls (n=176)
Gap-probe MISS rate	0.0%	gap_probe (n=329)

Outcome class counts

Class	Count
COV-CLEAN-CORRECT	8
COV-CLEAN-WRONG	168
COV-HIT-PLUS	2861
COV-WRONG	1184
GEN-INVALID	384

View 4 — Per-rule precision (M5-B)

Decomposes the catalog's package-level precision (= 0% on M5) into per-rule contributors. NC denominator: 176 evaluable negative_control rows. Sorted by NC FPR descending.

Rule	NC FPR	NC firings	Total firings	Targeted firings	Precision when fires
`W-ON-02`	89.8%	158	4203	180	4.3%
`COMPOUND-03`	23.9%	42	1606	180	11.2%
`W-AR-02`	23.9%	42	2552	194	7.6%
`COMPOUND-01`	21.6%	38	2713	110	4.1%
`W-CON-01`	19.9%	35	391	179	45.8%
`W-CON-04`	17.6%	31	313	177	56.5%
`W-EP-04`	2.8%	5	915	179	19.6%
`W-AR-01`	1.1%	2	172	170	98.8%
`W-AL-01`	0.0%	0	4045	178	4.4%
`W-AL-02`	0.0%	0	205	180	87.8%
`W-AR-03`	0.0%	0	178	178	100.0%
`W-AR-05`	0.0%	0	3902	59	1.5%
`W-CON-02`	0.0%	0	180	180	100.0%
`W-CON-05`	0.0%	0	179	179	100.0%
`W-EP-01`	0.0%	0	186	185	99.5%
`W-EP-02`	0.0%	0	3459	178	5.1%
`W-PROV-01`	0.0%	0	121	121	100.0%

Performance characterization (v1.8 D2)

Hardware: hardware spec not configured (set UOFA_HW_SPEC env var)

Metric	Value (ms)
Mean total_eval_ms	3921
Median total_eval_ms	3761
p95 total_eval_ms	6019
Sample size	4221

Slowest 20 packages

spec_id	variant	total_eval_ms
adv-2026-p2-022-compound-03_medium_morrison-cou1	20	9466
adv-2026-p2-021-compound-01_medium_morrison-cou2	14	9371
adv-2026-p2-021-compound-01_medium_morrison-cou2	15	9357
adv-2026-p2-021-compound-01_medium_morrison-cou2	12	9326
adv-2026-p2-021-compound-01_medium_morrison-cou2	13	9310
adv-2026-p2-021-compound-01_medium_morrison-cou2	11	9303
adv-2026-p2-021-compound-01_medium_morrison-cou2	5	9288
adv-2026-p2-022-compound-03_medium_morrison-cou1	18	9281
adv-2026-p2-021-compound-01_medium_morrison-cou2	2	9267
adv-2026-p2-021-compound-01_medium_morrison-cou2	4	9243
adv-2026-p2-021-compound-01_medium_morrison-cou2	3	9169
adv-2026-p2-022-compound-03_high_nagaraja-cou1	1	9155
adv-2026-p2-022-compound-03_medium_morrison-cou1	19	9040
adv-2026-p2-021-compound-01_medium_morrison-cou2	1	8776
adv-2026-p2-021-compound-01_low_morrison-cou2	3	8535
adv-2026-p2-021-compound-01_low_morrison-cou2	5	8516
adv-2026-p2-021-compound-01_low_morrison-cou2	2	8450
adv-2026-p2-021-compound-01_low_morrison-cou2	4	8440
adv-2026-p2-021-compound-01_low_morrison-cou2	1	8437
adv-2026-p2-022-compound-03_high_nagaraja-cou1	2	8417

n = 4605 package rows.