diff --git a/README.md b/README.md index 7e0a6e0..2e52f57 100644 --- a/README.md +++ b/README.md @@ -190,7 +190,29 @@ model such as `qwen3:8b` or `qwen3:14b`. Enable `Ollama profile advisor` and set `Profile advisor model` in Settings. Advisor output must be valid JSON and is validated against fields actually seen in the stream before it can be applied. Unknown fields, raw message fields, internal `fgai_*` fields, and unknown -detectors are rejected. +detectors are rejected. The advisor is only called for fields the deterministic +pass is genuinely unsure about (see below), not the whole profile every poll, and +is throttled by `Profile advisor interval seconds` so it does not run more often +than necessary on a memory-constrained local GPU. + +Field discovery does not rely only on field names. Every observed field is also +classified from its sample values into a shape (IP address, timestamp, port, +numeric, low-cardinality categorical, or free text) with a 0-100 confidence +score. A field name match is still used, but only as a small, capped nudge on +top of that shape evidence, so a vendor-specific field name SignalScope has +never seen before can still be recognized as an entity, categorical, or numeric +field from its values alone. Two IP-shaped fields with ambiguous names are +told apart by which one leans more private (source-like) versus more public +(destination-like). Detectors can also be suggested from observed value +patterns, for example a low-cardinality field containing values such as +`denied` or `blocked` suggesting the deny-action detector, independent of +what the field or stream happens to be named. `Field confidence threshold` in +Settings controls both how confident a field must be before it counts as +unambiguous (for advisor scoping) and, when auto-apply is enabled, before its +stream profile is applied automatically. Each recommended profile in the +dashboard status payload carries the underlying per-field shape and confidence +values, so a `field_confidence`/`field_shapes` map is available for any field +even when it was not selected into the applied profile. Windows-like streams are recognized from stream names such as `Windows`, `Winlog`, `Security Event Log`, `Sysmon`, and `Powershell`, or from Windows event @@ -246,6 +268,22 @@ categorical, numeric, and detector fields. Existing profile names, field weights and detector threshold settings are preserved, so this is the fast path after field-alias matching improves or after Graylog starts parsing additional fields. +Enable `Auto-apply high-confidence stream profiles` in Settings to skip the +`Apply`/`Update profile` click for streams where every selected field already +clears `Field confidence threshold`. A profile is only ever auto-applied when it +has no manually-set field: if even one field in an existing profile was set by +hand, that whole stream is left alone by auto-apply and stays on the normal +review flow. Fields written by auto-apply are marked internally so a later +manual edit through the Profile editor is never silently overwritten by a +future auto-apply cycle. If an auto-applied field's classification drifts +later, for example because a vendor changed their log format, SignalScope does +not silently re-apply a new mapping; it only records the drift in the status +payload (`profile_drift`) for now, with no dashboard panel yet, so the live +profile stays exactly as it was applied until someone reviews it. The auto-apply +and advisor-scoping thresholds share the same `Field confidence threshold` +setting, and drift-history retention is controlled separately by `Field +confidence history retention days`. + Profile discovery is accumulated over monitor cycles. This matters in high-EPS environments where each poll only fetches a raw sample for context while aggregate queries count the full window. Fields seen in earlier samples are kept diff --git a/src/fgai/dashboard.py b/src/fgai/dashboard.py index 9e96fa7..8723c5a 100644 --- a/src/fgai/dashboard.py +++ b/src/fgai/dashboard.py @@ -187,7 +187,7 @@ HTML = """

Events and Anomalies

Baseline and Stream Health

Operator Guidance

Waiting for monitor data.

Correlation Map

Waiting for correlated entities.

Investigation Incidents

Anomalies

Recommendations

AI Assessment

LLM assessment disabled.

Triage Queue

Field Baseline Deviations

Active Feedback Rules

Related Activity Across Sources

Block Candidates

Threat Intelligence

Policy Findings

Diagnostics

-

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

+

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

How To Use SignalScope

1. Normal workflow

  1. Settings: connect Graylog MCP and enable streams.
  2. Settings: apply recommended profiles for missing streams.
  3. Diagnostics: confirm raw samples, aggregate counts, and profile readiness.
  4. Overview: use Operator Guidance, incidents, trends, and correlation map.
  5. Findings: review only high-signal deviations first, then mark decisions.

2. Stream health

  • ready: profile exists, events are arriving, and baseline fields are ready.
  • learning: profile exists and events arrive, but baseline age or buckets are still too low.
  • missing_profile: stream is enabled but no profile exists. Apply or edit one.
  • no_events: stream is enabled but the current poll has no raw sample events.
  • partial_fetch: Graylog returned only part of the requested raw sample.

3. Ready fields

0/8 means 8 profile fields are tracked but none are mature enough yet. A field needs at least 12 baseline buckets and the configured Baseline training days before it is ready.

During learning, treat findings as signals to tune profiles, not as final alerts.

4. Profiles

  • Entity fields define who or what behavior is tracked, such as user, host, source IP, or application actor.
  • Baseline fields define the changing behavior to learn, such as action, event ID, service, URL, status, or counters.
  • Relationships learn pairs such as username to srcip or host to process.
  • Detectors add burst checks for auth failures, DNS queries, and deny actions.

5. Findings

  • Start with Triage Queue and incidents, not raw long tables.
  • Open evidence details before confirming a finding.
  • Use Expected for known behavior, False positive for bad signal, Confirmed for real investigation items.
  • Use expiry when a behavior is expected only temporarily.

6. High EPS / MCP

  • Use aggregate or auto fetch mode for high EPS streams.
  • Keep raw samples small enough for context; aggregate counts represent the full window.
  • Sample capped is normal in aggregate mode. Truncated raw mode means you may miss context.
  • If MCP is stale, the UI shows cached status so you can still inspect previous findings.

7. Ollama

  • Dashboard assessment summarizes current evidence.
  • Profile advisor maps unknown/custom fields and suggests relationships.
  • Ollama advice is constrained to fields discovered from Graylog; unknown fields are rejected.

8. Baseline DB size

  • Baseline DB size is the local SQLite behavior baseline.
  • Retention deletes old rows, but SQLite only returns disk space after manual VACUUM.
  • If the DB grows fast, lower bucket retention, value retention, or max values per entity field in Settings.
  • Stop the service before running signalscope baseline-maintenance --vacuum.

9. What to fix first

  1. No streams enabled.
  2. Enabled streams with missing profiles.
  3. Enabled streams with zero raw events.
  4. Profiles stuck at 0 ready fields after the training window.
  5. Too many repeated findings without review feedback.
  6. Baseline DB growing without maintenance.
@@ -1103,6 +1103,10 @@ async function applySuggestedProfile(streamId) { profile_advisor_enabled: Boolean(config.profile_advisor_enabled), profile_advisor_model: config.profile_advisor_model || 'qwen3:8b', profile_advisor_timeout: config.profile_advisor_timeout || 240, + profile_advisor_interval_seconds: config.profile_advisor_interval_seconds || 300, + field_confidence_threshold: config.field_confidence_threshold ?? 60, + field_confidence_retention_days: config.field_confidence_retention_days || 90, + auto_apply_profile_suggestions: Boolean(config.auto_apply_profile_suggestions), threat_intel_enabled: Boolean(config.threat_intel_enabled), threat_intel_provider: config.threat_intel_provider || 'auto', threat_intel_daily_limit: config.threat_intel_daily_limit || 100, diff --git a/tests/test_dashboard.py b/tests/test_dashboard.py index bf3e119..b6b9471 100644 --- a/tests/test_dashboard.py +++ b/tests/test_dashboard.py @@ -115,6 +115,13 @@ class DashboardTests(unittest.TestCase): self.assertIn('name="llm_timeout"', HTML) self.assertIn("llm_timeout: config.llm_timeout || 180", HTML) + def test_dashboard_exposes_field_confidence_and_auto_apply_settings(self): + self.assertIn('name="field_confidence_threshold"', HTML) + self.assertIn('name="profile_advisor_interval_seconds"', HTML) + self.assertIn('name="field_confidence_retention_days"', HTML) + self.assertIn('name="auto_apply_profile_suggestions"', HTML) + self.assertIn("field_confidence_threshold: config.field_confidence_threshold ?? 60", HTML) + if __name__ == "__main__": unittest.main()