Found live: a baseline-maintenance vacuum surfaced eventtime/timestamp
sitting in a stream's categorical_fields, hitting the max-values-per-field
cap at 2001 distinct values. Same failure mode as the original context
incident — a timestamp is near-unique on essentially every event, so
tracking it as a categorical or relationship field guarantees a "new"
finding every time. Extends _SHAPE_VETO (entity/categorical/relationship)
to also exclude "timestamp" shape, reusing the existing veto machinery and
its default 60-confidence threshold (timestamp classification isn't capped
low the way free_text is, so the stricter default applies correctly).
Existing profiles with this issue get flagged automatically via
detect_shape_mismatches, which already reuses the same veto check.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
classify_field_shape() received a deduplicated set of observed values with
no real event-volume context, so cardinality was always measured against
itself — trivially 100%, making any field with more than ~16 distinct
values look "high cardinality" no matter how large the real traffic volume
was. This misclassified genuinely low-cardinality fields (e.g. username: 82
distinct values across 1,601 events) as free text, which the new shape-veto
work then surfaced as false-positive "remove this field" warnings in
production. Fixed by passing real event coverage as an explicit
total_events parameter; omitting it keeps prior behavior for other callers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Completes the last deferred item from ROADMAP §0: entities recognized as
the same real-world concept across enabled streams (e.g. srcip in both a
firewall and an IIS stream) now share baseline knowledge of entity
presence/age and known relationship-target values, instead of each stream
learning it cold. Implemented additively (synthetic shared_entity:<group>
namespace in the existing baseline.py tables, no schema migration), scoped
to entity presence + relationship known-values only — per-stream
distribution baselines stay untouched. Also fixes a latent alias-map
collision bug found while building this (ALIAS_CANONICAL_BY_FIELD let
"context" silently steal "hostname"/"dns_query" from their own canonical
groups).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Field roles are now inferred from observed values instead of names,
with src/dst disambiguation, value-pattern detector hints, an Ollama
advisor scoped to ambiguous fields only, confidence-gated auto-apply
with manual-field protection, and drift flagging for review
