Add real detector health tooling: learning state and historical precision
Replaces the old Prometheus-only "detector precision" metric, which recomputed from whatever field_deviations happened to be active in the current poll cycle cross-referenced against live FeedbackStore rules — a revoked or expired suppression rule made the signal vanish, and nothing was tracked between polls. Adds DetectorPrecisionStore, a permanent append-only log of review outcomes (kept deliberately separate from FeedbackStore's mutable suppression rules) recorded the moment a human reviews a finding, plus BaselineStore.detector_readiness() mirroring the existing per-field profile_readiness() for per-detector learning state, and a plain-language detector reference. Surfaced in a new Detector Health dashboard panel. Observability only — nothing here changes scoring or suppression. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -104,6 +104,17 @@ class DashboardTests(unittest.TestCase):
|
||||
self.assertIn("Shared Entity Baselines", HTML)
|
||||
self.assertIn("shared_entity_baselines", HTML)
|
||||
|
||||
def test_dashboard_shows_detector_health(self):
|
||||
self.assertIn("detector-health", HTML)
|
||||
self.assertIn("Detector Health", HTML)
|
||||
self.assertIn("detector_health", HTML)
|
||||
|
||||
def test_review_actions_carry_detector_for_precision_tracking(self):
|
||||
self.assertIn('data-detector="${esc(r.detector', HTML)
|
||||
self.assertIn("detector: button.dataset.detector", HTML)
|
||||
self.assertIn("detector: pendingReview.detector", HTML)
|
||||
self.assertIn("detector: row.detector", HTML)
|
||||
|
||||
def test_dashboard_shows_shape_mismatch_warnings(self):
|
||||
self.assertIn('id="profileShapeWarnings"', HTML)
|
||||
self.assertIn("profile_shape_warnings", HTML)
|
||||
|
||||
Reference in New Issue
Block a user