diff --git a/README.md b/README.md index 92a2c9f..1d76642 100644 --- a/README.md +++ b/README.md @@ -186,7 +186,7 @@ stored under `graylog_stream_profiles` in `state/fgai-config.json`. The Settings page also shows recommended stream profiles built from observed field coverage and cardinality. These recommendations use deterministic discovery first, then can optionally be refined by a local Ollama profile advisor -model such as `qwen2.5:3b-instruct` (default) or a larger model on more capable +model such as `qwen3:8b` (default) or a smaller model on constrained hardware. Enable `Ollama profile advisor` and set `Profile advisor model` in Settings. Advisor output must be valid JSON and is validated against fields actually seen in the stream before it can be applied. @@ -672,27 +672,37 @@ fgai suggest-blocks --logs logs/fg_syslog.jsonl --execute --expiry-minutes 60 Optional local LLM summary through Ollama: ```bash -ollama pull qwen2.5:7b-instruct +ollama pull llama3.1 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm --llm-timeout 300 ``` -For slower machines, pick a smaller model instead of raising the timeout: +For slower machines or a memory-constrained GPU, pick a smaller model instead +of just raising the timeout: ```bash +ollama pull qwen2.5:3b-instruct OLLAMA_MODEL=qwen2.5:3b-instruct OLLAMA_TIMEOUT=300 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm ``` -**Sizing the model to your GPU**: the defaults (`qwen2.5:7b-instruct` for the -dashboard assessment, `qwen2.5:3b-instruct` for the profile advisor) target a -6-8 GB VRAM card at Q4 quantization with some headroom for context. On a -memory-constrained card (6 GB or less, e.g. a mobile/low-power GPU), drop both -to `qwen2.5:3b-instruct` or smaller — tight VRAM causes Ollama to spill onto -CPU, which shows up as slow or timed-out assessments rather than an obvious -out-of-memory error. On a workstation-class GPU with 16 GB+ VRAM, a larger -model such as `qwen3:14b` for the dashboard assessment will give noticeably -better analysis quality. The profile advisor only ever classifies a handful -of ambiguous fields per call (see the Graylog MCP section below), so it stays -small and fast even on constrained hardware without much quality loss. +**Sizing the model to your GPU**: the defaults (`llama3.1`, 8B, for the +dashboard assessment; `qwen3:8b` for the profile advisor) are a reasonable +general-purpose choice, but on a memory-constrained card (6 GB VRAM or less, +e.g. a mobile/low-power GPU) they can leave little headroom for context, +which shows up as slow or timed-out assessments rather than an obvious +out-of-memory error. If that happens, switch to a smaller model such as +`qwen2.5:3b-instruct` for both settings. On a workstation-class GPU with +16 GB+ VRAM, a larger model such as `qwen3:14b` for the dashboard assessment +will give noticeably better analysis quality. The profile advisor only ever +classifies a handful of ambiguous fields per call (see the Graylog MCP +section below), so it can stay small even on constrained hardware without +much quality loss. + +**Whichever model you choose, `ollama pull` it first** — Settings just holds +a model name string, and the dashboard doesn't pull anything on your behalf. +Pointing `Ollama model`/`Profile advisor model` at a tag you haven't pulled +locally makes every assessment fail silently rather than an obvious error. +Check `ollama list` (or the dashboard's own "Installed Ollama Models" panel +in Settings) before changing either setting. ## Optional FortiGate Input @@ -736,7 +746,7 @@ end - `FORTIGATE_VERIFY_TLS`: `true` or `false`, defaults to `true`. - `FGAI_ALLOWLIST`: comma-separated IPs/CIDRs never to block. - `OLLAMA_HOST`: defaults to `http://127.0.0.1:11434`. -- `OLLAMA_MODEL`: defaults to `qwen2.5:7b-instruct`. Size this to your GPU's VRAM — see the sizing note above. +- `OLLAMA_MODEL`: defaults to `llama3.1`. Size this to your GPU's VRAM — see the sizing note above. - `OLLAMA_TIMEOUT`: Ollama request timeout in seconds, defaults to `180`. - `FGAI_LLM`: set to `1` to enable dashboard Ollama analyst notes. - `FGAI_LLM_INTERVAL`: seconds between dashboard LLM notes, defaults to `300`. diff --git a/src/fgai/config.py b/src/fgai/config.py index 48d3309..ac9ee74 100644 --- a/src/fgai/config.py +++ b/src/fgai/config.py @@ -28,7 +28,7 @@ DEFAULT_CONFIG: dict[str, object] = { "llm_model": "", "llm_timeout": 180, "profile_advisor_enabled": False, - "profile_advisor_model": "qwen2.5:3b-instruct", + "profile_advisor_model": "qwen3:8b", "profile_advisor_timeout": 240, "profile_advisor_interval_seconds": 300, "field_confidence_threshold": 60, diff --git a/src/fgai/dashboard.py b/src/fgai/dashboard.py index 82397b7..dce26a3 100644 --- a/src/fgai/dashboard.py +++ b/src/fgai/dashboard.py @@ -193,7 +193,7 @@ HTML = """

Events and Anomalies

Baseline and Stream Health

Operator Guidance

Waiting for monitor data.

Correlation Map

Waiting for correlated entities.

Investigation Incidents

Anomalies

Recommendations

AI Assessment

LLM assessment disabled.

Triage Queue

Field Baseline Deviations

Active Feedback Rules

Related Activity Across Sources
Block Candidates
Threat Intelligence
Policy Findings

Diagnostics

-

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

+

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

How To Use SignalScope

1. Normal workflow

  1. Settings: connect Graylog MCP and enable streams.
  2. Settings: apply recommended profiles for missing streams.
  3. Diagnostics: confirm raw samples, aggregate counts, and profile readiness.
  4. Overview: use Operator Guidance, incidents, trends, and correlation map.
  5. Findings: review only high-signal deviations first, then mark decisions.

2. Stream health

  • ready: profile exists, events are arriving, and baseline fields are ready.
  • learning: profile exists and events arrive, but baseline age or buckets are still too low.
  • missing_profile: stream is enabled but no profile exists. Apply or edit one.
  • no_events: stream is enabled but the current poll has no raw sample events.
  • partial_fetch: Graylog returned only part of the requested raw sample.

3. Ready fields

0/8 means 8 profile fields are tracked but none are mature enough yet. A field needs at least 12 baseline buckets and the configured Baseline training days before it is ready.

During learning, treat findings as signals to tune profiles, not as final alerts.

4. Profiles

  • Entity fields define who or what behavior is tracked, such as user, host, source IP, or application actor.
  • Baseline fields define the changing behavior to learn, such as action, event ID, service, URL, status, or counters.
  • Relationships learn pairs such as username to srcip or host to process.
  • Detectors add burst checks for auth failures, DNS queries, and deny actions.

5. Findings

  • Start with Triage Queue and incidents, not raw long tables.
  • Open evidence details before confirming a finding.
  • Use Expected for known behavior, False positive for bad signal, Confirmed for real investigation items.
  • Use expiry when a behavior is expected only temporarily.

6. High EPS / MCP

  • Use aggregate or auto fetch mode for high EPS streams.
  • Keep raw samples small enough for context; aggregate counts represent the full window.
  • Sample capped is normal in aggregate mode. Truncated raw mode means you may miss context.
  • If MCP is stale, the UI shows cached status so you can still inspect previous findings.

7. Ollama

  • Dashboard assessment summarizes current evidence.
  • Profile advisor maps unknown/custom fields and suggests relationships.
  • Ollama advice is constrained to fields discovered from Graylog; unknown fields are rejected.

8. Baseline DB size

  • Baseline DB size is the local SQLite behavior baseline.
  • Retention deletes old rows, but SQLite only returns disk space after manual VACUUM.
  • If the DB grows fast, lower bucket retention, value retention, or max values per entity field in Settings.
  • Stop the service before running signalscope baseline-maintenance --vacuum.

9. What to fix first

  1. No streams enabled.
  2. Enabled streams with missing profiles.
  3. Enabled streams with zero raw events.
  4. Profiles stuck at 0 ready fields after the training window.
  5. Too many repeated findings without review feedback.
  6. Baseline DB growing without maintenance.
@@ -1280,7 +1280,7 @@ async function applySuggestedProfile(streamId) { llm_model: config.llm_model || '', llm_timeout: config.llm_timeout || 180, profile_advisor_enabled: Boolean(config.profile_advisor_enabled), - profile_advisor_model: config.profile_advisor_model || 'qwen2.5:3b-instruct', + profile_advisor_model: config.profile_advisor_model || 'qwen3:8b', profile_advisor_timeout: config.profile_advisor_timeout || 240, profile_advisor_interval_seconds: config.profile_advisor_interval_seconds || 300, field_confidence_threshold: config.field_confidence_threshold ?? 60, diff --git a/src/fgai/llm.py b/src/fgai/llm.py index 7c0d9a3..158bf43 100644 --- a/src/fgai/llm.py +++ b/src/fgai/llm.py @@ -41,7 +41,7 @@ def ollama_summary( timeout: int | None = None, ) -> str: host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/") - selected_model = model or os.getenv("OLLAMA_MODEL", "qwen2.5:7b-instruct") + selected_model = model or os.getenv("OLLAMA_MODEL", "llama3.1") selected_timeout = timeout or int(os.getenv("OLLAMA_TIMEOUT", "180")) prompt = { "analysis": analysis or {}, @@ -378,7 +378,7 @@ def ollama_profile_advice( ambiguous_fields_by_stream: dict[str, list[str]] | None = None, ) -> list[dict[str, object]]: host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/") - selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen2.5:3b-instruct") + selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen3:8b") selected_timeout = timeout or int(os.getenv("FGAI_PROFILE_ADVISOR_TIMEOUT", "240")) scoped = bool(ambiguous_fields_by_stream) compact = [