Add real detector health tooling: learning state and historical precision

Replaces the old Prometheus-only "detector precision" metric, which
recomputed from whatever field_deviations happened to be active in the
current poll cycle cross-referenced against live FeedbackStore rules — a
revoked or expired suppression rule made the signal vanish, and nothing was
tracked between polls. Adds DetectorPrecisionStore, a permanent append-only
log of review outcomes (kept deliberately separate from FeedbackStore's
mutable suppression rules) recorded the moment a human reviews a finding,
plus BaselineStore.detector_readiness() mirroring the existing per-field
profile_readiness() for per-detector learning state, and a plain-language
detector reference. Surfaced in a new Detector Health dashboard panel.
Observability only — nothing here changes scoring or suppression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-24 16:22:02 +02:00
parent b0cc9dbdfe
commit 36516faf9a
11 changed files with 315 additions and 10 deletions

View File

@@ -182,6 +182,16 @@ The review buttons are part of the detection loop:
- Expiry should be used for temporary expected changes, such as maintenance or a
migration window.
Every review click also permanently records which detector produced the
finding and the verdict, independent of the suppression rule it creates —
so revoking or letting a rule expire later never erases that history. Check
`Diagnostics -> Detector Health` to see, per detector: whether it has
learned enough baseline history to be trusted yet, its confirmed-vs-false-positive
precision over the last 30 days, and a one-line description of what
actually triggers it. A detector with no reviews yet shows `n/a` for
precision rather than a misleading 0% or 100% — the number only becomes
meaningful once there's real history behind it.
## Primary Workflow: Graylog MCP
Graylog 7.1 MCP is the primary log-source integration. In the dashboard, open