diff --git a/README.md b/README.md index 2e52f57..92a2c9f 100644 --- a/README.md +++ b/README.md @@ -186,7 +186,8 @@ stored under `graylog_stream_profiles` in `state/fgai-config.json`. The Settings page also shows recommended stream profiles built from observed field coverage and cardinality. These recommendations use deterministic discovery first, then can optionally be refined by a local Ollama profile advisor -model such as `qwen3:8b` or `qwen3:14b`. Enable `Ollama profile advisor` and set +model such as `qwen2.5:3b-instruct` (default) or a larger model on more capable +hardware. Enable `Ollama profile advisor` and set `Profile advisor model` in Settings. Advisor output must be valid JSON and is validated against fields actually seen in the stream before it can be applied. Unknown fields, raw message fields, internal `fgai_*` fields, and unknown @@ -671,16 +672,28 @@ fgai suggest-blocks --logs logs/fg_syslog.jsonl --execute --expiry-minutes 60 Optional local LLM summary through Ollama: ```bash -ollama pull llama3.3 +ollama pull qwen2.5:7b-instruct fgai analyze-logs --logs logs/fg_syslog.jsonl --llm --llm-timeout 300 ``` -For slower machines or large models: +For slower machines, pick a smaller model instead of raising the timeout: ```bash -OLLAMA_MODEL=llama3.1 OLLAMA_TIMEOUT=300 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm +OLLAMA_MODEL=qwen2.5:3b-instruct OLLAMA_TIMEOUT=300 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm ``` +**Sizing the model to your GPU**: the defaults (`qwen2.5:7b-instruct` for the +dashboard assessment, `qwen2.5:3b-instruct` for the profile advisor) target a +6-8 GB VRAM card at Q4 quantization with some headroom for context. On a +memory-constrained card (6 GB or less, e.g. a mobile/low-power GPU), drop both +to `qwen2.5:3b-instruct` or smaller — tight VRAM causes Ollama to spill onto +CPU, which shows up as slow or timed-out assessments rather than an obvious +out-of-memory error. On a workstation-class GPU with 16 GB+ VRAM, a larger +model such as `qwen3:14b` for the dashboard assessment will give noticeably +better analysis quality. The profile advisor only ever classifies a handful +of ambiguous fields per call (see the Graylog MCP section below), so it stays +small and fast even on constrained hardware without much quality loss. + ## Optional FortiGate Input ## Synthetic Windows Test Input @@ -723,7 +736,7 @@ end - `FORTIGATE_VERIFY_TLS`: `true` or `false`, defaults to `true`. - `FGAI_ALLOWLIST`: comma-separated IPs/CIDRs never to block. - `OLLAMA_HOST`: defaults to `http://127.0.0.1:11434`. -- `OLLAMA_MODEL`: defaults to `llama3.1`. +- `OLLAMA_MODEL`: defaults to `qwen2.5:7b-instruct`. Size this to your GPU's VRAM — see the sizing note above. - `OLLAMA_TIMEOUT`: Ollama request timeout in seconds, defaults to `180`. - `FGAI_LLM`: set to `1` to enable dashboard Ollama analyst notes. - `FGAI_LLM_INTERVAL`: seconds between dashboard LLM notes, defaults to `300`. @@ -743,3 +756,70 @@ The agent separates detection from enforcement: - Source IPs must be globally routable and outside the allowlist. - Blocking requires `--execute`. - The FortiGate API call is limited to the quarantine/banned user monitor endpoint. + +## Production Deployment + +`start.sh` is convenient for a single machine you're logged into, but has no +supervision (a crashed process just stays dead) and no boot-time start. + +### Running under systemd + +Unit files for the monitor, dashboard, and optional local syslog listener are +in `systemd/`, with full setup and update instructions in +[`systemd/README.md`](systemd/README.md). They mirror `start.sh`'s exact +process arguments, add `Restart=on-failure`, and run under a dedicated +`signalscope` system user with `ProtectSystem=strict`/`ProtectHome=true` +sandboxing rather than as root. + +### Backup and Restore + +Everything SignalScope needs to restore — runtime configuration, feedback +rules, behavior baselines, incident state, and caches — lives under `state/`. +Nothing else needs backing up; logs and exports are reproducible/disposable. + +```bash +scripts/backup_state.sh # writes to ./backups/signalscope-state-.tar.gz +scripts/backup_state.sh /path/to/backups # or a custom destination +``` + +SQLite databases (`fgai-baseline.sqlite3`, `signalscope-history.sqlite3`, +`signalscope-status-cache.sqlite3`) are backed up with `sqlite3 .backup`, +which takes a consistent snapshot even while the monitor is actively writing +to them, rather than a plain file copy that can capture a torn/inconsistent +write mid-transaction. JSON files (`fgai-config.json`, +`signalscope-feedback.json`, `signalscope-incidents.json`, +`threat-intel-cache.json`) are already written atomically by SignalScope +(temp file + rename) and are copied directly. + +To restore: stop the monitor and dashboard, extract the archive's contents +over `state/`, then restart. + +### Exposing the Dashboard Beyond Loopback + +The dashboard has **no built-in authentication** — its entire access-control +model is "only accept connections from `127.0.0.1`" (`start.sh`'s and the +systemd unit's default `--host 127.0.0.1`). API routes that change state +(`POST /api/config`, `/api/feedback`, `/api/feedback/bulk`, incident actions, +and more) additionally check that the client's socket address is loopback, +regardless of what `Host` header a request carries. + +If you need to reach the dashboard from another machine, do not change +`--host` to `0.0.0.0` directly — put an authenticating reverse proxy in front +of it instead and keep SignalScope itself bound to loopback. For example, +with Caddy: + +``` +signalscope.internal.example.com { + basicauth { + analyst $2a$14$... # generate with: caddy hash-password + } + reverse_proxy 127.0.0.1:8088 +} +``` + +Or nginx with `auth_basic` / an `auth_request` subrequest to whatever SSO the +rest of your infrastructure already uses. Either way, terminate TLS at the +proxy and keep the token/credential prompts outside SignalScope's own process +— since state-changing routes only trust the *proxy's* loopback connection, +not the original client, the proxy is the only thing standing between a +public request and dashboard write access once `--host` is opened up. diff --git a/scripts/backup_state.sh b/scripts/backup_state.sh new file mode 100755 index 0000000..5b41db0 --- /dev/null +++ b/scripts/backup_state.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Backs up everything SignalScope needs to restore its configuration, +# feedback rules, baselines, and incident history: the whole state/ +# directory. SQLite databases are backed up live via `sqlite3 .backup` +# (a consistent snapshot even while the monitor is writing to them) +# instead of a plain file copy, which can capture a torn/inconsistent +# write. JSON files (config, feedback, incidents, caches) are already +# written atomically by SignalScope (temp file + rename), so a plain +# copy of those is safe. + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +STATE_DIR="${FGAI_STATE_DIR:-$ROOT_DIR/state}" +BACKUP_ROOT="${1:-$ROOT_DIR/backups}" +STAMP="$(date -u +%Y%m%dT%H%M%SZ)" +DEST_DIR="$BACKUP_ROOT/signalscope-state-$STAMP" + +if [ ! -d "$STATE_DIR" ]; then + printf 'State directory not found: %s\n' "$STATE_DIR" >&2 + exit 1 +fi + +mkdir -p "$DEST_DIR" + +shopt -s nullglob +for db in "$STATE_DIR"/*.sqlite3; do + name="$(basename "$db")" + if command -v sqlite3 >/dev/null 2>&1; then + sqlite3 "$db" ".backup '$DEST_DIR/$name'" + else + printf 'Warning: sqlite3 CLI not found, falling back to plain copy for %s (may be inconsistent if the monitor is running)\n' "$name" >&2 + cp "$db" "$DEST_DIR/$name" + fi +done + +for json_file in "$STATE_DIR"/*.json; do + cp "$json_file" "$DEST_DIR/$(basename "$json_file")" +done +shopt -u nullglob + +ARCHIVE="$BACKUP_ROOT/signalscope-state-$STAMP.tar.gz" +tar -czf "$ARCHIVE" -C "$BACKUP_ROOT" "signalscope-state-$STAMP" +rm -rf "$DEST_DIR" + +printf 'Backed up %s to %s\n' "$STATE_DIR" "$ARCHIVE" diff --git a/src/fgai/config.py b/src/fgai/config.py index ef075c2..48d3309 100644 --- a/src/fgai/config.py +++ b/src/fgai/config.py @@ -28,11 +28,12 @@ DEFAULT_CONFIG: dict[str, object] = { "llm_model": "", "llm_timeout": 180, "profile_advisor_enabled": False, - "profile_advisor_model": "qwen3:8b", + "profile_advisor_model": "qwen2.5:3b-instruct", "profile_advisor_timeout": 240, "profile_advisor_interval_seconds": 300, "field_confidence_threshold": 60, "field_confidence_retention_days": 90, + "feedback_generalization_min_values": 3, "auto_apply_profile_suggestions": False, "threat_intel_enabled": False, "threat_intel_provider": "auto", @@ -89,6 +90,11 @@ class ConfigStore: current[key] = min(100, max(0, int(value))) except (TypeError, ValueError): continue + elif key == "feedback_generalization_min_values": + try: + current[key] = min(20, max(2, int(value))) + except (TypeError, ValueError): + continue elif key == "threat_intel_provider" and value in {"auto", "abuseipdb", "virustotal"}: current[key] = value elif key == "graylog_streams" and isinstance(value, list): diff --git a/src/fgai/dashboard.py b/src/fgai/dashboard.py index 62bc22a..0a25802 100644 --- a/src/fgai/dashboard.py +++ b/src/fgai/dashboard.py @@ -191,9 +191,9 @@ HTML = """

Events and Anomalies

Baseline and Stream Health

Operator Guidance

Waiting for monitor data.

Correlation Map

Waiting for correlated entities.

Investigation Incidents

Anomalies

Recommendations

AI Assessment

LLM assessment disabled.
-

Triage Queue

Field Baseline Deviations

Active Feedback Rules

Related Activity Across Sources
Block Candidates
Threat Intelligence
Policy Findings
+

Triage Queue

Field Baseline Deviations

Active Feedback Rules

Related Activity Across Sources
Block Candidates
Threat Intelligence
Policy Findings

Diagnostics

-

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

+

Recommended Stream Profiles

Waiting for observed stream data.

Installed Ollama Models

Loading local Ollama models.

Runtime Configuration

How To Use SignalScope

1. Normal workflow

  1. Settings: connect Graylog MCP and enable streams.
  2. Settings: apply recommended profiles for missing streams.
  3. Diagnostics: confirm raw samples, aggregate counts, and profile readiness.
  4. Overview: use Operator Guidance, incidents, trends, and correlation map.
  5. Findings: review only high-signal deviations first, then mark decisions.

2. Stream health

  • ready: profile exists, events are arriving, and baseline fields are ready.
  • learning: profile exists and events arrive, but baseline age or buckets are still too low.
  • missing_profile: stream is enabled but no profile exists. Apply or edit one.
  • no_events: stream is enabled but the current poll has no raw sample events.
  • partial_fetch: Graylog returned only part of the requested raw sample.

3. Ready fields

0/8 means 8 profile fields are tracked but none are mature enough yet. A field needs at least 12 baseline buckets and the configured Baseline training days before it is ready.

During learning, treat findings as signals to tune profiles, not as final alerts.

4. Profiles

  • Entity fields define who or what behavior is tracked, such as user, host, source IP, or application actor.
  • Baseline fields define the changing behavior to learn, such as action, event ID, service, URL, status, or counters.
  • Relationships learn pairs such as username to srcip or host to process.
  • Detectors add burst checks for auth failures, DNS queries, and deny actions.

5. Findings

  • Start with Triage Queue and incidents, not raw long tables.
  • Open evidence details before confirming a finding.
  • Use Expected for known behavior, False positive for bad signal, Confirmed for real investigation items.
  • Use expiry when a behavior is expected only temporarily.

6. High EPS / MCP

  • Use aggregate or auto fetch mode for high EPS streams.
  • Keep raw samples small enough for context; aggregate counts represent the full window.
  • Sample capped is normal in aggregate mode. Truncated raw mode means you may miss context.
  • If MCP is stale, the UI shows cached status so you can still inspect previous findings.

7. Ollama

  • Dashboard assessment summarizes current evidence.
  • Profile advisor maps unknown/custom fields and suggests relationships.
  • Ollama advice is constrained to fields discovered from Graylog; unknown fields are rejected.

8. Baseline DB size

  • Baseline DB size is the local SQLite behavior baseline.
  • Retention deletes old rows, but SQLite only returns disk space after manual VACUUM.
  • If the DB grows fast, lower bucket retention, value retention, or max values per entity field in Settings.
  • Stop the service before running signalscope baseline-maintenance --vacuum.

9. What to fix first

  1. No streams enabled.
  2. Enabled streams with missing profiles.
  3. Enabled streams with zero raw events.
  4. Profiles stuck at 0 ready fields after the training window.
  5. Too many repeated findings without review feedback.
  6. Baseline DB growing without maintenance.
@@ -816,15 +816,27 @@ async function refresh() { if (fpRatio <= 0.2) return `likely worth confirming (${Math.round((1 - fpRatio) * 100)}% of ${total} reviewed)`; return esc('mixed history'); }; + const suggestPriority = r => { + if (r.feedback) return 1; + const stats = detectorPrecision[r.detector || 'unknown']; + const total = stats ? stats.confirmed + stats.false_positive : 0; + if (total < 3) return 1; + const fpRatio = stats.false_positive / total; + if (fpRatio <= 0.2) return 0; + if (fpRatio >= 0.7) return 2; + return 1; + }; const showReviewed = document.getElementById('showReviewedFindings')?.checked; const showLow = document.getElementById('showLowFindings')?.checked; const fieldRows = sourceFieldRows .filter(row => showReviewed || !['expected','false_positive'].includes(row.feedback || '')) .filter(row => showLow || Number(row.score || 0) >= 35 || row.feedback === 'confirmed') - .sort((left, right) => Number(right.score || 0) - Number(left.score || 0)) + .sort((left, right) => suggestPriority(left) - suggestPriority(right) || Number(right.score || 0) - Number(left.score || 0)) .slice(0, 50); document.getElementById('actionPlan').innerHTML = buildActionPlan({configuration, mcp, enabledStreams, streamsMissingProfile, pendingProfileSuggestions, displayedRawEvents, displayedAggregateEvents, fieldRows, incidents: data.incidents || [], correlations, baseline}); + const worthConfirmingCount = fieldRows.filter(row => suggestPriority(row) === 0).length; document.getElementById('findingSummary').textContent = `${fieldRows.length} shown from ${sourceFieldRows.length} raw deviations. Reviewed and low-score findings are hidden by default.`; + document.getElementById('findingSummaryBadge').textContent = worthConfirmingCount ? `(${fieldRows.length} shown, ${worthConfirmingCount} likely worth confirming)` : `(${fieldRows.length} shown)`; if (rawFieldRows.length) uiCache.fieldRows = rawFieldRows; document.getElementById('fieldDeviations').innerHTML = `${fieldRowsCached ? '

No current field deviations in this poll; showing cached findings from the last non-empty poll.

' : ''}` + table(fieldRows, [ {label:'', render:r => { const key = deviationKey(r); return ``; }}, @@ -948,6 +960,24 @@ async function refresh() { document.getElementById('feedbackNotice').textContent = response.ok ? 'Rule revoked.' : 'Could not revoke rule.'; refresh(); })); + const generalizationSuggestions = data.feedback_generalization_suggestions || []; + const generalizationContainer = document.getElementById('feedbackGeneralizationSuggestions'); + generalizationContainer.innerHTML = generalizationSuggestions.map(s => { + const samples = s.sample_values.map(esc).join(', '); + const more = s.value_count > s.sample_values.length ? `, +${s.value_count - s.sample_values.length} more` : ''; + return `

${s.value_count} ${esc(s.status)} rules for ${esc(s.entity)}/${esc(s.field)} on ${esc(s.stream_id)} (${samples}${more}) —

`; + }).join(''); + document.querySelectorAll('.merge-feedback-generalization').forEach(button => button.addEventListener('click', async () => { + const s = JSON.parse(button.dataset.payload); + if (!confirm(`Merge ${s.value_count} ${s.status} rules for ${s.entity}/${s.field} on ${s.stream_id} into one field-wide rule? The ${s.value_count} narrow rules will be revoked.`)) return; + const addResponse = await fetch('/api/feedback', {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({status: s.status, scope: 'field', entity: s.entity, stream_id: s.stream_id, field: s.field, note: `merged from ${s.value_count} value-scope rules`, author: lastReviewAuthor, expires_at: 0})}); + if (!addResponse.ok) { document.getElementById('feedbackNotice').textContent = 'Could not create merged rule.'; return; } + for (const id of s.entry_ids) { + await fetch('/api/feedback/revoke', {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({id})}); + } + document.getElementById('feedbackNotice').textContent = `Merged ${s.value_count} rules for ${s.entity}/${s.field}.`; + refresh(); + })); const relatedGroups = compactRelatedActivity(relatedRows); document.getElementById('relatedActivity').innerHTML = `

${relatedGroups.length} grouped rows shown from ${relatedRows.length} raw related events.

` + table(relatedGroups, [ {label:'Entity', key:'entity', render:r => entityCell(r)}, @@ -1196,11 +1226,12 @@ async function applySuggestedProfile(streamId) { llm_model: config.llm_model || '', llm_timeout: config.llm_timeout || 180, profile_advisor_enabled: Boolean(config.profile_advisor_enabled), - profile_advisor_model: config.profile_advisor_model || 'qwen3:8b', + profile_advisor_model: config.profile_advisor_model || 'qwen2.5:3b-instruct', profile_advisor_timeout: config.profile_advisor_timeout || 240, profile_advisor_interval_seconds: config.profile_advisor_interval_seconds || 300, field_confidence_threshold: config.field_confidence_threshold ?? 60, field_confidence_retention_days: config.field_confidence_retention_days || 90, + feedback_generalization_min_values: config.feedback_generalization_min_values || 3, auto_apply_profile_suggestions: Boolean(config.auto_apply_profile_suggestions), threat_intel_enabled: Boolean(config.threat_intel_enabled), threat_intel_provider: config.threat_intel_provider || 'auto', diff --git a/src/fgai/feedback.py b/src/fgai/feedback.py index d48646b..2bcb8bb 100644 --- a/src/fgai/feedback.py +++ b/src/fgai/feedback.py @@ -175,3 +175,36 @@ def feedback_matches(entry: dict[str, object], *, entity: str, stream_id: str, f def select_match(candidates: list[dict[str, object]]) -> dict[str, object]: return min(candidates, key=lambda item: SCOPE_PRIORITY.get(str(item.get("scope") or _resolve_scope(item)), 1)) + + +def suggest_generalizations( + entries: list[dict[str, object]], *, min_values: int = 3, sample_limit: int = 5 +) -> list[dict[str, object]]: + """Find groups of narrow value-scope rules a human could merge into one field-scope rule. + + Only ever looks at entries a human already created (value scope) — never + reasons about unseen values, so it can't pre-judge a new anomaly. + """ + groups: dict[tuple[str, str, str, str], list[dict[str, object]]] = {} + for entry in entries: + scope = str(entry.get("scope") or _resolve_scope(entry)) + if scope != "value": + continue + status = str(entry.get("status", "")) + if status not in VALID_STATUSES: + continue + key = (str(entry.get("stream_id", "")), str(entry.get("entity", "")), str(entry.get("field", "")), status) + groups.setdefault(key, []).append(entry) + + suggestions = [] + for (stream_id, entity, field, status), group in groups.items(): + distinct_values = sorted({str(item.get("value", "")) for item in group}) + if len(distinct_values) < min_values: + continue + suggestions.append({ + "stream_id": stream_id, "entity": entity, "field": field, "status": status, + "value_count": len(distinct_values), "sample_values": distinct_values[:sample_limit], + "entry_ids": [str(item.get("id", "")) for item in group], + }) + suggestions.sort(key=lambda item: -item["value_count"]) + return suggestions diff --git a/src/fgai/llm.py b/src/fgai/llm.py index 158bf43..7c0d9a3 100644 --- a/src/fgai/llm.py +++ b/src/fgai/llm.py @@ -41,7 +41,7 @@ def ollama_summary( timeout: int | None = None, ) -> str: host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/") - selected_model = model or os.getenv("OLLAMA_MODEL", "llama3.1") + selected_model = model or os.getenv("OLLAMA_MODEL", "qwen2.5:7b-instruct") selected_timeout = timeout or int(os.getenv("OLLAMA_TIMEOUT", "180")) prompt = { "analysis": analysis or {}, @@ -378,7 +378,7 @@ def ollama_profile_advice( ambiguous_fields_by_stream: dict[str, list[str]] | None = None, ) -> list[dict[str, object]]: host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/") - selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen3:8b") + selected_model = model or os.getenv("FGAI_PROFILE_ADVISOR_MODEL", "qwen2.5:3b-instruct") selected_timeout = timeout or int(os.getenv("FGAI_PROFILE_ADVISOR_TIMEOUT", "240")) scoped = bool(ambiguous_fields_by_stream) compact = [ diff --git a/src/fgai/monitor.py b/src/fgai/monitor.py index e37517a..9eb3bb4 100644 --- a/src/fgai/monitor.py +++ b/src/fgai/monitor.py @@ -12,7 +12,7 @@ from .baseline import BaselineStore from .config import ConfigStore from .correlation import correlate_source_ips from .event_context import build_event_context -from .feedback import FeedbackStore, SUPPRESSING_STATUSES, feedback_matches, select_match +from .feedback import FeedbackStore, SUPPRESSING_STATUSES, feedback_matches, select_match, suggest_generalizations from .graylog_aggregate import GraylogAggregateSource from .graylog_mcp import GraylogMcpClient from .graylog_source import GraylogStreamSource @@ -524,6 +524,8 @@ def build_status( deviation["feedback_rule_id"] = match.get("id", "") if match["status"] in SUPPRESSING_STATUSES: deviation["score"] = 0 + generalization_min_values = int(runtime_values.get("feedback_generalization_min_values", 3) or 3) + feedback_generalization_suggestions = suggest_generalizations(feedback, min_values=generalization_min_values) anomalies = detect_source_anomalies(events, limit=anomaly_limit, baselines=profiles, field_deviations=field_deviations) baseline_events = baseline.ingest(events) if baseline else 0 profile_baseline_fields = ( @@ -691,6 +693,7 @@ def build_status( "triage_queue": triage_queue, "sequence_findings": sequence_findings, "feedback": feedback, + "feedback_generalization_suggestions": feedback_generalization_suggestions, "cross_source_correlations": correlations, "incidents": incidents, "data_quality": assess_data_quality(events, mcp_status), diff --git a/systemd/README.md b/systemd/README.md new file mode 100644 index 0000000..ea850c8 --- /dev/null +++ b/systemd/README.md @@ -0,0 +1,68 @@ +# Running SignalScope under systemd + +These units replace `start.sh`/`stop.sh` with proper service supervision +(auto-restart, journald logging, boot-time start). They assume the repo lives +at `/opt/signalscope` and runs as a dedicated `signalscope` user — adjust the +paths and `User=`/`Group=` lines if you deploy elsewhere. + +## One-time setup + +```bash +sudo useradd --system --home-dir /opt/signalscope --shell /usr/sbin/nologin signalscope +sudo mkdir -p /opt/signalscope +sudo cp -r . /opt/signalscope/ # from a checkout of this repo +sudo chown -R signalscope:signalscope /opt/signalscope + +# Create the venv and install dependencies as the service user +sudo -u signalscope python3 -m venv /opt/signalscope/.venv +sudo -u signalscope /opt/signalscope/.venv/bin/pip install --upgrade pip +sudo -u signalscope /opt/signalscope/.venv/bin/pip install -e /opt/signalscope + +sudo cp systemd/*.service /etc/systemd/system/ +sudo systemctl daemon-reload +``` + +## Enable and start + +```bash +sudo systemctl enable --now signalscope-monitor.service +sudo systemctl enable --now signalscope-dashboard.service + +# Only if you use the local syslog input instead of Graylog MCP: +sudo systemctl enable --now signalscope-syslog.service +``` + +## Everyday operations + +```bash +systemctl status signalscope-monitor signalscope-dashboard +journalctl -u signalscope-monitor -f +journalctl -u signalscope-dashboard -f +sudo systemctl restart signalscope-monitor signalscope-dashboard +``` + +Configure Graylog MCP, streams, and profiles the same way as always: open the +dashboard and use Settings — configuration lives in +`/opt/signalscope/state/fgai-config.json` regardless of how the process was +started. + +## Updating + +```bash +sudo systemctl stop signalscope-dashboard signalscope-monitor +sudo -u signalscope git -C /opt/signalscope pull +sudo -u signalscope /opt/signalscope/.venv/bin/pip install -e /opt/signalscope +sudo systemctl start signalscope-monitor signalscope-dashboard +``` + +## Notes + +- `signalscope-dashboard.service` binds `127.0.0.1` by default, matching the + documented safety default in the main README. See "Exposing the dashboard + beyond loopback" there before changing `--host` to anything else. +- `state/` (config, feedback rules, baselines, incidents) is the only + directory that needs backing up — see `scripts/backup_state.sh` and the + "Backup and Restore" section in the main README. +- The monitor unit's `ExecStart` has a commented example for enabling the + Ollama dashboard assessment (`--llm ...`); the profile advisor is a + dashboard Settings toggle and needs no unit-file change. diff --git a/systemd/signalscope-dashboard.service b/systemd/signalscope-dashboard.service new file mode 100644 index 0000000..3c77c3e --- /dev/null +++ b/systemd/signalscope-dashboard.service @@ -0,0 +1,29 @@ +[Unit] +Description=SignalScope dashboard +After=network-online.target signalscope-monitor.service +Wants=network-online.target + +[Service] +Type=simple +User=signalscope +Group=signalscope +WorkingDirectory=/opt/signalscope +# Binds to loopback only by default. Do not change --host to 0.0.0.0 without +# putting an authenticating reverse proxy in front — see the README section +# "Exposing the dashboard beyond loopback". +ExecStart=/opt/signalscope/.venv/bin/fgai dashboard \ + --host 127.0.0.1 \ + --port 8088 \ + --status-file /opt/signalscope/state/fgai-status.json \ + --config-file /opt/signalscope/state/fgai-config.json \ + --image-dir /opt/signalscope/images +Restart=on-failure +RestartSec=5 +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/opt/signalscope/state + +[Install] +WantedBy=multi-user.target diff --git a/systemd/signalscope-monitor.service b/systemd/signalscope-monitor.service new file mode 100644 index 0000000..9a1a6a5 --- /dev/null +++ b/systemd/signalscope-monitor.service @@ -0,0 +1,30 @@ +[Unit] +Description=SignalScope continuous monitor +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=signalscope +Group=signalscope +WorkingDirectory=/opt/signalscope +# To enable the local Ollama dashboard assessment, add to ExecStart below: +# --llm --llm-interval 300 --llm-timeout 180 --model qwen2.5:7b-instruct +ExecStart=/opt/signalscope/.venv/bin/fgai monitor \ + --logs /opt/signalscope/logs/fg_syslog.jsonl \ + --output /opt/signalscope/state/fgai-status.json \ + --interval 10 \ + --baseline-db /opt/signalscope/state/fgai-baseline.sqlite3 \ + --history-db /opt/signalscope/state/signalscope-history.sqlite3 \ + --status-cache-db /opt/signalscope/state/signalscope-status-cache.sqlite3 \ + --config-file /opt/signalscope/state/fgai-config.json +Restart=on-failure +RestartSec=5 +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/opt/signalscope/state /opt/signalscope/logs /opt/signalscope/run + +[Install] +WantedBy=multi-user.target diff --git a/systemd/signalscope-syslog.service b/systemd/signalscope-syslog.service new file mode 100644 index 0000000..0b53ad1 --- /dev/null +++ b/systemd/signalscope-syslog.service @@ -0,0 +1,30 @@ +[Unit] +Description=SignalScope local syslog listener (optional — only needed for the local_syslog log source, not for Graylog MCP) +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=signalscope +Group=signalscope +WorkingDirectory=/opt/signalscope +ExecStart=/opt/signalscope/.venv/bin/fgai listen-syslog \ + --host 0.0.0.0 \ + --port 5514 \ + --output /opt/signalscope/logs/fg_syslog.jsonl \ + --rotate-bytes 26214400 \ + --rotate-count 14 \ + --quiet +Restart=on-failure +RestartSec=5 +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/opt/signalscope/logs +# To listen on the standard syslog port 514 instead of 5514, change --port +# above to 514 and uncomment the line below instead of running as root: +# AmbientCapabilities=CAP_NET_BIND_SERVICE + +[Install] +WantedBy=multi-user.target diff --git a/tests/test_config.py b/tests/test_config.py index 1d73633..1dd3858 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -82,6 +82,16 @@ class ConfigTests(unittest.TestCase): public = store.update({"field_confidence_threshold": "not-a-number"}) self.assertEqual(public["field_confidence_threshold"], 70) + def test_feedback_generalization_min_values_is_numeric_and_clamped(self): + with tempfile.TemporaryDirectory() as directory: + store = ConfigStore(str(Path(directory) / "config.json")) + public = store.update({"feedback_generalization_min_values": "1"}) + self.assertEqual(public["feedback_generalization_min_values"], 2) + public = store.update({"feedback_generalization_min_values": "500"}) + self.assertEqual(public["feedback_generalization_min_values"], 20) + public = store.update({"feedback_generalization_min_values": "5"}) + self.assertEqual(public["feedback_generalization_min_values"], 5) + def test_empty_stream_update_does_not_clear_existing_streams(self): with tempfile.TemporaryDirectory() as directory: store = ConfigStore(str(Path(directory) / "config.json")) diff --git a/tests/test_dashboard.py b/tests/test_dashboard.py index 1dc3b0c..5db0cd5 100644 --- a/tests/test_dashboard.py +++ b/tests/test_dashboard.py @@ -94,6 +94,35 @@ class DashboardTests(unittest.TestCase): self.assertIn("openDetailIds", HTML) self.assertIn("addEventListener('toggle'", HTML) + def test_dashboard_field_deviations_are_collapsible_and_open_by_default(self): + start = HTML.index("Field Baseline Deviations") + preceding = HTML[max(0, start - 80):start] + self.assertIn('
', preceding) + self.assertIn('id="findingSummaryBadge"', HTML) + + def test_dashboard_sorts_likely_worth_confirming_rows_first(self): + self.assertIn('const suggestPriority', HTML) + self.assertIn('suggestPriority(left) - suggestPriority(right)', HTML) + + def test_dashboard_has_generalization_suggestion_panel(self): + self.assertIn('id="feedbackGeneralizationSuggestions"', HTML) + + def test_dashboard_generalization_merge_button_confirms_before_widening_scope(self): + self.assertIn('merge-feedback-generalization', HTML) + self.assertIn('Merge into one field-wide rule', HTML) + self.assertIn('will be revoked', HTML) + + def test_dashboard_generalization_merge_only_ever_posts_field_scope(self): + start = HTML.index("querySelectorAll('.merge-feedback-generalization')") + end = HTML.index('}));', start) + merge_handler = HTML[start:end] + self.assertIn("scope: 'field'", merge_handler) + self.assertNotIn("scope: 'entity'", merge_handler) + self.assertNotIn("scope: 'stream'", merge_handler) + + def test_dashboard_exposes_feedback_generalization_setting(self): + self.assertIn('name="feedback_generalization_min_values"', HTML) + def test_dashboard_suggests_review_from_detector_history(self): self.assertIn("label:'Suggested', render:suggestReview", HTML) self.assertIn("likely false positive", HTML) diff --git a/tests/test_feedback.py b/tests/test_feedback.py index ab2d8bf..1121354 100644 --- a/tests/test_feedback.py +++ b/tests/test_feedback.py @@ -3,7 +3,7 @@ import time import unittest from pathlib import Path -from fgai.feedback import FeedbackStore, feedback_matches, select_match +from fgai.feedback import FeedbackStore, feedback_matches, select_match, suggest_generalizations class FeedbackTests(unittest.TestCase): @@ -120,6 +120,43 @@ class FeedbackTests(unittest.TestCase): self.assertEqual(select_match(candidates), value_rule) self.assertEqual(select_match([stream_rule, entity_rule]), entity_rule) + def test_suggest_generalizations_groups_narrow_value_rules_by_stream_entity_field_status(self): + entries = [ + {"scope": "value", "status": "false_positive", "entity": "alice", "stream_id": "windows", "field": "username", "value": "bob", "id": "1"}, + {"scope": "value", "status": "false_positive", "entity": "alice", "stream_id": "windows", "field": "username", "value": "carol", "id": "2"}, + {"scope": "value", "status": "false_positive", "entity": "alice", "stream_id": "windows", "field": "username", "value": "dave", "id": "3"}, + {"scope": "value", "status": "confirmed", "entity": "alice", "stream_id": "windows", "field": "username", "value": "eve", "id": "4"}, + ] + suggestions = suggest_generalizations(entries, min_values=3) + self.assertEqual(len(suggestions), 1) + self.assertEqual(suggestions[0]["value_count"], 3) + self.assertEqual(suggestions[0]["status"], "false_positive") + self.assertEqual(set(suggestions[0]["entry_ids"]), {"1", "2", "3"}) + + def test_suggest_generalizations_respects_min_values_threshold(self): + entries = [ + {"scope": "value", "status": "expected", "entity": "a", "stream_id": "s", "field": "f", "value": "x", "id": "1"}, + {"scope": "value", "status": "expected", "entity": "a", "stream_id": "s", "field": "f", "value": "y", "id": "2"}, + ] + self.assertEqual(suggest_generalizations(entries, min_values=3), []) + + def test_suggest_generalizations_ignores_broader_scope_entries(self): + entries = [ + {"scope": "field", "status": "false_positive", "entity": "a", "stream_id": "s", "field": "f", "id": "1"}, + {"scope": "entity", "status": "expected", "entity": "a", "id": "2"}, + ] + self.assertEqual(suggest_generalizations(entries, min_values=2), []) + + def test_suggest_generalizations_keeps_different_entities_separate(self): + entries = [ + {"scope": "value", "status": "false_positive", "entity": "alice", "stream_id": "s", "field": "f", "value": "1", "id": "1"}, + {"scope": "value", "status": "false_positive", "entity": "alice", "stream_id": "s", "field": "f", "value": "2", "id": "2"}, + {"scope": "value", "status": "false_positive", "entity": "bob", "stream_id": "s", "field": "f", "value": "3", "id": "3"}, + ] + self.assertEqual(suggest_generalizations(entries, min_values=2), [ + {"stream_id": "s", "entity": "alice", "field": "f", "status": "false_positive", "value_count": 2, "sample_values": ["1", "2"], "entry_ids": ["1", "2"]} + ]) + if __name__ == "__main__": unittest.main()