fix isssue classification and doc

README.md

@@ -190,7 +190,29 @@ model such as `qwen3:8b` or `qwen3:14b`. Enable `Ollama profile advisor` and set
 `Profile advisor model` in Settings. Advisor output must be valid JSON and is
 validated against fields actually seen in the stream before it can be applied.
 Unknown fields, raw message fields, internal `fgai_*` fields, and unknown
-detectors are rejected.
+detectors are rejected. The advisor is only called for fields the deterministic
+pass is genuinely unsure about (see below), not the whole profile every poll, and
+is throttled by `Profile advisor interval seconds` so it does not run more often
+than necessary on a memory-constrained local GPU.
+
+Field discovery does not rely only on field names. Every observed field is also
+classified from its sample values into a shape (IP address, timestamp, port,
+numeric, low-cardinality categorical, or free text) with a 0-100 confidence
+score. A field name match is still used, but only as a small, capped nudge on
+top of that shape evidence, so a vendor-specific field name SignalScope has
+never seen before can still be recognized as an entity, categorical, or numeric
+field from its values alone. Two IP-shaped fields with ambiguous names are
+told apart by which one leans more private (source-like) versus more public
+(destination-like). Detectors can also be suggested from observed value
+patterns, for example a low-cardinality field containing values such as
+`denied` or `blocked` suggesting the deny-action detector, independent of
+what the field or stream happens to be named. `Field confidence threshold` in
+Settings controls both how confident a field must be before it counts as
+unambiguous (for advisor scoping) and, when auto-apply is enabled, before its
+stream profile is applied automatically. Each recommended profile in the
+dashboard status payload carries the underlying per-field shape and confidence
+values, so a `field_confidence`/`field_shapes` map is available for any field
+even when it was not selected into the applied profile.
 
 Windows-like streams are recognized from stream names such as `Windows`,
 `Winlog`, `Security Event Log`, `Sysmon`, and `Powershell`, or from Windows event
@@ -246,6 +268,22 @@ categorical, numeric, and detector fields. Existing profile names, field weights
 and detector threshold settings are preserved, so this is the fast path after
 field-alias matching improves or after Graylog starts parsing additional fields.
 
+Enable `Auto-apply high-confidence stream profiles` in Settings to skip the
+`Apply`/`Update profile` click for streams where every selected field already
+clears `Field confidence threshold`. A profile is only ever auto-applied when it
+has no manually-set field: if even one field in an existing profile was set by
+hand, that whole stream is left alone by auto-apply and stays on the normal
+review flow. Fields written by auto-apply are marked internally so a later
+manual edit through the Profile editor is never silently overwritten by a
+future auto-apply cycle. If an auto-applied field's classification drifts
+later, for example because a vendor changed their log format, SignalScope does
+not silently re-apply a new mapping; it only records the drift in the status
+payload (`profile_drift`) for now, with no dashboard panel yet, so the live
+profile stays exactly as it was applied until someone reviews it. The auto-apply
+and advisor-scoping thresholds share the same `Field confidence threshold`
+setting, and drift-history retention is controlled separately by `Field
+confidence history retention days`.
+
 Profile discovery is accumulated over monitor cycles. This matters in high-EPS
 environments where each poll only fetches a raw sample for context while
 aggregate queries count the full window. Fields seen in earlier samples are kept

tests/test_dashboard.py

@@ -115,6 +115,13 @@ class DashboardTests(unittest.TestCase):
         self.assertIn('name="llm_timeout"', HTML)
         self.assertIn("llm_timeout: config.llm_timeout || 180", HTML)
 
+    def test_dashboard_exposes_field_confidence_and_auto_apply_settings(self):
+        self.assertIn('name="field_confidence_threshold"', HTML)
+        self.assertIn('name="profile_advisor_interval_seconds"', HTML)
+        self.assertIn('name="field_confidence_retention_days"', HTML)
+        self.assertIn('name="auto_apply_profile_suggestions"', HTML)
+        self.assertIn("field_confidence_threshold: config.field_confidence_threshold ?? 60", HTML)
+
 
 if __name__ == "__main__":
     unittest.main()