Commit Graph

15 Commits

Author SHA256 Message Date
larssand
e61403d7f6 Surface baseline maintenance status in the dashboard
Retention and per-field value capping already ran automatically every poll
cycle, but the dashboard showed only a single "DB size" line — no
visibility into what got deleted, which fields are hitting the
max-values-per-field cap, or whether disk space is reclaimable, without
SSHing in. baseline.py's maintenance() now also reports the specific
(stream, entity, field) rows it trimmed for exceeding the cap, surfaced in
a new Baseline Maintenance dashboard panel alongside size/reclaimable bytes
and rows deleted last cycle. Deliberately no live-vacuum button: VACUUM
needs the service stopped first, so the dashboard only shows when it's
worth doing and points at the existing offline CLI command.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-24 16:35:09 +02:00
larssand
36516faf9a Add real detector health tooling: learning state and historical precision
Replaces the old Prometheus-only "detector precision" metric, which
recomputed from whatever field_deviations happened to be active in the
current poll cycle cross-referenced against live FeedbackStore rules — a
revoked or expired suppression rule made the signal vanish, and nothing was
tracked between polls. Adds DetectorPrecisionStore, a permanent append-only
log of review outcomes (kept deliberately separate from FeedbackStore's
mutable suppression rules) recorded the moment a human reviews a finding,
plus BaselineStore.detector_readiness() mirroring the existing per-field
profile_readiness() for per-detector learning state, and a plain-language
detector reference. Surfaced in a new Detector Health dashboard panel.
Observability only — nothing here changes scoring or suppression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-24 16:22:02 +02:00
larssand
83ffef3570 Pool baseline state across streams sharing an entity concept
Completes the last deferred item from ROADMAP §0: entities recognized as
the same real-world concept across enabled streams (e.g. srcip in both a
firewall and an IIS stream) now share baseline knowledge of entity
presence/age and known relationship-target values, instead of each stream
learning it cold. Implemented additively (synthetic shared_entity:<group>
namespace in the existing baseline.py tables, no schema migration), scoped
to entity presence + relationship known-values only — per-stream
distribution baselines stay untouched. Also fixes a latent alias-map
collision bug found while building this (ALIAS_CANONICAL_BY_FIELD let
"context" silently steal "hostname"/"dns_query" from their own canonical
groups).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 18:50:14 +02:00
larssand
d314c94dee fix monitor error 2026-07-06 13:02:10 +02:00
larssand
841f94dc9f fix retention 2026-07-06 12:41:34 +02:00
larssand
7d0e66c239 add relationship fields 2026-07-02 15:07:15 +02:00
larssand
d540b8a77d Implemented cleanup/triage direction. ui, and baselinbe days 2026-06-29 20:20:00 +02:00
larssand
d764c5a038 Implemented the baseline/noise and SQLite growth improvements. 2026-06-29 19:58:49 +02:00
larssand
68370217da Fortsatte roadmapen med multi-entity stream profiles 2026-06-29 19:08:23 +02:00
larssand
9da1aedf52 configurable per-field detector weights är klart. 2026-06-25 19:36:08 +02:00
larssand
ea4aaf57ed Continued the Detection Quality roadmap. 2026-06-24 21:32:18 +02:00
larssand
cdef3e1355 Continued the Detection Quality roadmap milestone. 2026-06-24 21:14:07 +02:00
larssand
868022008a Implemented the next multi-source detection layer in this repository. 2026-06-24 19:16:16 +02:00
larssand
5bb381a063 fix ui 2026-06-21 15:19:24 +02:00
larssand
d41d318134 add baseline and sqlite 2026-06-21 15:04:23 +02:00