fix ollama

This commit is contained in:
larssand
2026-09-23 13:40:51 +02:00
parent a3f545f1b0
commit 5f8b1cd098
4 changed files with 30 additions and 20 deletions

View File

@@ -186,7 +186,7 @@ stored under `graylog_stream_profiles` in `state/fgai-config.json`.
The Settings page also shows recommended stream profiles built from observed
field coverage and cardinality. These recommendations use deterministic
discovery first, then can optionally be refined by a local Ollama profile advisor
model such as `qwen2.5:3b-instruct` (default) or a larger model on more capable
model such as `qwen3:8b` (default) or a smaller model on constrained
hardware. Enable `Ollama profile advisor` and set
`Profile advisor model` in Settings. Advisor output must be valid JSON and is
validated against fields actually seen in the stream before it can be applied.
@@ -672,27 +672,37 @@ fgai suggest-blocks --logs logs/fg_syslog.jsonl --execute --expiry-minutes 60
Optional local LLM summary through Ollama:
```bash
ollama pull qwen2.5:7b-instruct
ollama pull llama3.1
fgai analyze-logs --logs logs/fg_syslog.jsonl --llm --llm-timeout 300
```
For slower machines, pick a smaller model instead of raising the timeout:
For slower machines or a memory-constrained GPU, pick a smaller model instead
of just raising the timeout:
```bash
ollama pull qwen2.5:3b-instruct
OLLAMA_MODEL=qwen2.5:3b-instruct OLLAMA_TIMEOUT=300 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm
```
**Sizing the model to your GPU**: the defaults (`qwen2.5:7b-instruct` for the
dashboard assessment, `qwen2.5:3b-instruct` for the profile advisor) target a
6-8 GB VRAM card at Q4 quantization with some headroom for context. On a
memory-constrained card (6 GB or less, e.g. a mobile/low-power GPU), drop both
to `qwen2.5:3b-instruct` or smaller — tight VRAM causes Ollama to spill onto
CPU, which shows up as slow or timed-out assessments rather than an obvious
out-of-memory error. On a workstation-class GPU with 16 GB+ VRAM, a larger
model such as `qwen3:14b` for the dashboard assessment will give noticeably
better analysis quality. The profile advisor only ever classifies a handful
of ambiguous fields per call (see the Graylog MCP section below), so it stays
small and fast even on constrained hardware without much quality loss.
**Sizing the model to your GPU**: the defaults (`llama3.1`, 8B, for the
dashboard assessment; `qwen3:8b` for the profile advisor) are a reasonable
general-purpose choice, but on a memory-constrained card (6 GB VRAM or less,
e.g. a mobile/low-power GPU) they can leave little headroom for context,
which shows up as slow or timed-out assessments rather than an obvious
out-of-memory error. If that happens, switch to a smaller model such as
`qwen2.5:3b-instruct` for both settings. On a workstation-class GPU with
16 GB+ VRAM, a larger model such as `qwen3:14b` for the dashboard assessment
will give noticeably better analysis quality. The profile advisor only ever
classifies a handful of ambiguous fields per call (see the Graylog MCP
section below), so it can stay small even on constrained hardware without
much quality loss.
**Whichever model you choose, `ollama pull` it first** — Settings just holds
a model name string, and the dashboard doesn't pull anything on your behalf.
Pointing `Ollama model`/`Profile advisor model` at a tag you haven't pulled
locally makes every assessment fail silently rather than an obvious error.
Check `ollama list` (or the dashboard's own "Installed Ollama Models" panel
in Settings) before changing either setting.
## Optional FortiGate Input
@@ -736,7 +746,7 @@ end
- `FORTIGATE_VERIFY_TLS`: `true` or `false`, defaults to `true`.
- `FGAI_ALLOWLIST`: comma-separated IPs/CIDRs never to block.
- `OLLAMA_HOST`: defaults to `http://127.0.0.1:11434`.
- `OLLAMA_MODEL`: defaults to `qwen2.5:7b-instruct`. Size this to your GPU's VRAM — see the sizing note above.
- `OLLAMA_MODEL`: defaults to `llama3.1`. Size this to your GPU's VRAM — see the sizing note above.
- `OLLAMA_TIMEOUT`: Ollama request timeout in seconds, defaults to `180`.
- `FGAI_LLM`: set to `1` to enable dashboard Ollama analyst notes.
- `FGAI_LLM_INTERVAL`: seconds between dashboard LLM notes, defaults to `300`.

View File

@@ -28,7 +28,7 @@ DEFAULT_CONFIG: dict[str, object] = {
"llm_model": "",
"llm_timeout": 180,
"profile_advisor_enabled": False,
"profile_advisor_model": "qwen2.5:3b-instruct",
"profile_advisor_model": "qwen3:8b",
"profile_advisor_timeout": 240,
"profile_advisor_interval_seconds": 300,
"field_confidence_threshold": 60,

File diff suppressed because one or more lines are too long

View File

@@ -41,7 +41,7 @@ def ollama_summary(
timeout: int | None = None,
) -> str:
host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/")
selected_model = model or os.getenv("OLLAMA_MODEL", "qwen2.5:7b-instruct")
selected_model = model or os.getenv("OLLAMA_MODEL", "llama3.1")
selected_timeout = timeout or int(os.getenv("OLLAMA_TIMEOUT", "180"))
prompt = {
"analysis": analysis or {},
@@ -378,7 +378,7 @@ def ollama_profile_advice(
ambiguous_fields_by_stream: dict[str, list[str]] | None = None,
) -> list[dict[str, object]]:
host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/")
selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen2.5:3b-instruct")
selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen3:8b")
selected_timeout = timeout or int(os.getenv("FGAI_PROFILE_ADVISOR_TIMEOUT", "240"))
scoped = bool(ambiguous_fields_by_stream)
compact = [