add new structure

ROADMAP.md

@@ -29,18 +29,18 @@ parts of sections 1 and 6 below (baseline confidence tooling and compact
 incident cards both depend on findings being clustered correctly first) — do
 this section before spending more effort on either.
 
-- [ ] Validate name-matched fields against observed value shape too, not just
+- [x] Validate name-matched fields against observed value shape too, not just
   fields that had no name match. A field that matches a trusted name (e.g.
   `context`) but is actually free-text (a full raw log line, confirmed in 12
   of 25 audited stream profiles) must not be trusted just because its name
   matched a known-useful alias.
-- [ ] Make relationship-candidate selection shape-aware: never propose a
+- [x] Make relationship-candidate selection shape-aware: never propose a
   free-text-shaped field as a relationship target, regardless of how it
   entered `categorical_fields`. This alone was responsible for 17 of 25
   entries in one real triage queue snapshot (91 individual `new_relationship`
   deviations from a single misconfigured `*->context` pairing, repeated
   across 5 streams).
-- [ ] Re-validate existing stored profiles' `relationship_fields` and
+- [x] Re-validate existing stored profiles' `relationship_fields` and
   `categorical_fields` against current observed shapes on an ongoing basis,
   not just at profile-creation time — flag (never silently rewrite) fields
   that no longer look fit for their assigned role, and surface a one-click
@@ -50,18 +50,22 @@ this section before spending more effort on either.
   (for example a source IP seen in both a firewall stream and an IIS
   stream), instead of a cold, isolated baseline per stream per field. A value
   already known-normal in one contributing stream must not register as "new"
-  the first time a different, related stream happens to see it.
-- [ ] Cluster field deviations by `(detector, field, value-pattern)`
-  signature *before* scoring, not after. A pattern recurring identically
-  across many entities must surface as one clustered item with an occurrence
-  count and a "how many entities" breakdown, not N independently-scored
-  "critical" rows that all have to be discovered, selected, and bulk-cleared
-  by hand after the fact.
-- [ ] Recalibrate severity so cluster size is itself a signal: a pattern
-  hitting many unrelated entities identically is usually a
-  configuration/detector problem, not N separate critical incidents, and
-  should score at or below an isolated occurrence, never above it.
-- [ ] Update README usage documentation alongside each item above as it
+  the first time a different, related stream happens to see it. Deferred:
+  needs a `baseline.py` schema change and its own dedicated planning pass.
+- [x] Cluster field deviations by `(detector, field)` signature *before*
+  scoring, not after (value-pattern was dropped from the original signature —
+  detector+field alone already matched the real incident's cross-stream
+  pattern, and adding value-pattern would have under-clustered near-identical
+  but not-byte-identical raw messages). A pattern recurring identically
+  across many entities surfaces as one clustered item with an occurrence
+  count (`cluster_size`) shown inline in the Evidence column, not N
+  independently-scored "critical" rows that all have to be discovered,
+  selected, and bulk-cleared by hand after the fact.
+- [x] Recalibrate severity so cluster size is itself a signal: a pattern
+  hitting five or more entities identically is capped at score 25 regardless
+  of its uncapped score, so it never outranks an isolated, uncapped
+  occurrence.
+- [x] Update README usage documentation alongside each item above as it
   ships — a behavior change is not done until it is documented.
 
 Acceptance: replaying the dataset that produced the original 25-entry,