Public/fgAI
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pool baseline state across streams sharing an entity concept

Browse Source 
Completes the last deferred item from ROADMAP §0: entities recognized as
the same real-world concept across enabled streams (e.g. srcip in both a
firewall and an IIS stream) now share baseline knowledge of entity
presence/age and known relationship-target values, instead of each stream
learning it cold. Implemented additively (synthetic shared_entity:<group>
namespace in the existing baseline.py tables, no schema migration), scoped
to entity presence + relationship known-values only — per-stream
distribution baselines stay untouched. Also fixes a latent alias-map
collision bug found while building this (ALIAS_CANONICAL_BY_FIELD let
"context" silently steal "hostname"/"dns_query" from their own canonical
groups).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-23 18:50:14 +02:00
parent 8c155ec9cf
commit 83ffef3570
12 changed files with 305 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
README.md
View File

@@ -289,6 +289,23 @@ If the Ollama advisor returns no usable profile for a stream, the row stays on
the deterministic profile and is labeled as a heuristic fallback instead of
pretending the whole recommendation failed.
When two or more enabled streams recognize the same real-world entity
concept through this alias grouping (for example `srcip` used as the entity
field in both a firewall stream and an IIS stream), SignalScope
automatically pools part of their baseline state instead of learning it
independently per stream: an entity's presence/age history and its known
relationship-target values are shared across the pool, so a value already
established as normal in one contributing stream does not get reported as
"new" the first time a different, related stream happens to observe it.
This is automatic — there is nothing to configure — and it only ever
*reduces* false "already new" flags for values genuinely seen elsewhere in
the pool; a value that is actually new everywhere is still reported exactly
as before. Per-stream statistical baselines (event-rate bursts, detector
bursts, numeric-field distributions) are not pooled and stay independent per
stream, since those distributions legitimately differ by log source.
Active pools are listed under `Diagnostics -> Shared Entity Baselines`,
showing which entity concept is shared and which streams contribute to it.
Recommended profiles can be re-applied to existing profiles. Enable `show
existing profiles` and click `Update profile` to append newly discovered entity,
categorical, numeric, and detector fields. Existing profile names, field weights,