Public/fgAI
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Also veto timestamp-shaped fields from categorical/entity/relationship roles

Browse Source 
Found live: a baseline-maintenance vacuum surfaced eventtime/timestamp
sitting in a stream's categorical_fields, hitting the max-values-per-field
cap at 2001 distinct values. Same failure mode as the original context
incident — a timestamp is near-unique on essentially every event, so
tracking it as a categorical or relationship field guarantees a "new"
finding every time. Extends _SHAPE_VETO (entity/categorical/relationship)
to also exclude "timestamp" shape, reusing the existing veto machinery and
its default 60-confidence threshold (timestamp classification isn't capped
low the way free_text is, so the stricter default applies correctly).
Existing profiles with this issue get flagged automatically via
detect_shape_mismatches, which already reuses the same veto check.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-24 18:27:45 +02:00
parent e61403d7f6
commit bc4d2f465f
3 changed files with 35 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
README.md
View File

@@ -252,12 +252,16 @@ even when it was not selected into the applied profile.
A name match never overrides value shape, even for names on the trusted
priority lists such as `context`. If the observed values for a name-matched
field confidently look like free text (a raw log line or URL, not a
low-cardinality category) it is excluded from entity, categorical, and
relationship-target selection regardless of its name, both for a fresh
recommendation and as a second, independent check when a field reaches
relationship-candidate selection through cross-stream field sharing. A field
with too few samples to classify is never excluded on that basis alone; the
veto only fires on a confident, contradicting shape.
low-cardinality category) or like a timestamp, it is excluded from entity,
categorical, and relationship-target selection regardless of its name, both
for a fresh recommendation and as a second, independent check when a field
reaches relationship-candidate selection through cross-stream field sharing.
The timestamp case matters for the same reason free text does: a timestamp
is near-unique on essentially every event, so tracking it as a categorical
or relationship field is guaranteed to produce a "never seen before" finding
every single time, exactly the failure mode that made `context` a problem.
A field with too few samples to classify is never excluded on that basis
alone; the veto only fires on a confident, contradicting shape.
Windows-like streams are recognized from stream names such as `Windows`,
`Winlog`, `Security Event Log`, `Sysmon`, and `Powershell`, or from Windows event