Add real detector health tooling: learning state and historical precision

Replaces the old Prometheus-only "detector precision" metric, which
recomputed from whatever field_deviations happened to be active in the
current poll cycle cross-referenced against live FeedbackStore rules — a
revoked or expired suppression rule made the signal vanish, and nothing was
tracked between polls. Adds DetectorPrecisionStore, a permanent append-only
log of review outcomes (kept deliberately separate from FeedbackStore's
mutable suppression rules) recorded the moment a human reviews a finding,
plus BaselineStore.detector_readiness() mirroring the existing per-field
profile_readiness() for per-detector learning state, and a plain-language
detector reference. Surfaced in a new Detector Health dashboard panel.
Observability only — nothing here changes scoring or suppression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-24 16:22:02 +02:00
parent b0cc9dbdfe
commit 36516faf9a
11 changed files with 315 additions and 10 deletions

View File

@@ -104,6 +104,17 @@ class DashboardTests(unittest.TestCase):
self.assertIn("Shared Entity Baselines", HTML)
self.assertIn("shared_entity_baselines", HTML)
def test_dashboard_shows_detector_health(self):
self.assertIn("detector-health", HTML)
self.assertIn("Detector Health", HTML)
self.assertIn("detector_health", HTML)
def test_review_actions_carry_detector_for_precision_tracking(self):
self.assertIn('data-detector="${esc(r.detector', HTML)
self.assertIn("detector: button.dataset.detector", HTML)
self.assertIn("detector: pendingReview.detector", HTML)
self.assertIn("detector: row.detector", HTML)
def test_dashboard_shows_shape_mismatch_warnings(self):
self.assertIn('id="profileShapeWarnings"', HTML)
self.assertIn("profile_shape_warnings", HTML)