fgAI/ROADMAP.md

SignalScope Roadmap

This roadmap tracks the path from the current local Graylog analytics agent to a production-ready, multi-source security analytics tool. Items are ordered by operational value and dependency, not by UI appeal.

Current Foundation

• Graylog MCP connection, stream discovery, field discovery, and stream selection.
• Per-stream profiles for entity, timestamp, categorical, and numeric fields.
• Local five-minute field baselines with duplicate-event protection.
• Time-aware baseline comparison using matching UTC weekday/hour when available.
• Generic entity correlation for IP addresses, users, and hostnames.
• Centralized common field alias normalization for firewall, DNS, proxy, endpoint, and web logs.
• Cross-stream correlation timelines and investigation incident grouping.
• Field-deviation review: expected, false positive, confirmed, note, and expiry.
• Local Ollama analyst assessment with incident and feedback context.
• Cached AbuseIPDB/VirusTotal enrichment with a daily request budget.
• Prometheus-compatible low-cardinality metrics endpoint.
• Local dashboard for status, findings, incidents, diagnostics, and configuration.

1. Detection Quality

Goal: make findings more accurate before adding more integrations.

• Add baseline confidence based on sample count and time-bucket coverage.
• Add generic event-rate burst detection per stream/entity.
• Add rare-value detection with a minimum historical observation threshold.
• Add detector-specific authentication failure, DNS volume, and denied-traffic burst thresholds.
• Add configurable per-field detector weights.
• Add sequence detection, for example DNS lookup -> outbound connection -> authentication event.
• Add per-stream detector enablement and thresholds in the UI.
• Add a dry-run replay command for historic JSONL or Graylog exports using temporary baselines.
• Add direct Graylog MCP time-range replay and result comparison against saved detector configurations.
• Add dashboard controls for launching safe replay jobs and viewing detector deltas.

Acceptance: each finding shows its detector, confidence, baseline sample count, current value, expected value, and a bounded set of raw-event references.

2. Entity and Incident Investigation

Goal: make one incident answer what happened, to whom, and across which sources.

• Allow multiple entity fields per stream, such as user plus source IP plus hostname.
• Add entity aliasing: map DHCP, VPN, DNS, and endpoint identities to the same host where evidence supports it.
• Add configurable incident grouping windows and incident lifecycle: open, acknowledged, resolved, reopened.
• Persist incident state and analyst notes separately from transient detection output.
• Add direct Graylog query links or query details for each timeline event.
• Add investigation export as JSON and Markdown report.

Acceptance: an analyst can open an incident, see an ordered multi-stream timeline, review evidence, and record an outcome without losing it after the next monitor poll.

3. Feedback and Learning Controls

Goal: reduce repeat noise without hiding genuinely new behavior.

• Show active feedback rules, expiry, scope, author, and matching findings in the UI.
• Support feedback scope choices: exact value, entity and field, entire entity, or stream-wide rule.
• Require an expiry for broad suppressions and warn before a broad rule is saved.
• Track detector precision using confirmed versus false-positive feedback.
• Include feedback-derived precision and suppression counts in metrics and diagnostics.

Acceptance: a false-positive decision suppresses only the intended pattern and all suppressions are auditable and reversible.

4. Threat Intelligence and Enrichment

Goal: enrich public indicators predictably without exhausting provider quotas.

• Make cache TTL and daily budget configurable in the dashboard.
• Add provider health, cache hit ratio, and quota/budget visibility.
• Add domain and URL enrichment behind separate provider budgets.
• Keep enrichment asynchronous so a slow provider never delays monitoring.
• Store enrichment provenance and observation timestamps with incident evidence.

Acceptance: no refresh cycle causes uncontrolled external lookups, and every reputation result shows provider, timestamp, and cache state.

5. Production Operations

Goal: run reliably in a monitored environment.

• Add systemd unit files for monitor, dashboard, and optional local syslog listener.
• Add health and readiness endpoints with last successful Graylog fetch time.
• Add structured application logs and configurable retention for status/history/baseline data.
• Add backup and migration procedure for SQLite state.
• Add Checkmk local-check output in addition to Prometheus metrics.
• Add authentication/reverse-proxy guidance before exposing the dashboard beyond loopback.

Acceptance: deployment, restart, upgrade, backup, and monitoring have documented, repeatable procedures.

6. Dashboard Usability

Goal: optimize the UI for security investigation rather than raw tables.

• Separate stream enablement from one-profile-at-a-time profile editing.
• Add stream coverage diagnostics for enabled/profiled/ready/no-event streams.
• Replace remaining long tables with compact incident and entity cards where appropriate.
• Add incident filters for stream, severity, entity type, review state, and time range.
• Add baseline versus current charts per selected entity and field.
• Add an interactive entity graph backed by actual correlations.
• Preserve user view state, filters, expanded evidence, and selected tab across refreshes.
• Add visual data-quality warnings for missing timestamps, entity fields, or truncated Graylog results.

Acceptance: common triage can be completed from the dashboard without manually parsing raw JSON or searching for stream IDs.

7. Integrations and Extensibility

Goal: add log sources and outputs without adding source-specific logic everywhere.

• Define versioned stream-profile templates for FortiGate, Windows, DNS/AdGuard, Nginx, Squid, VPN, and Proxmox.
• Add inventory-style stream coverage to guide which streams need profiles before templates are added.
• Add common alias normalizer so new firewall/proxy/DNS streams can populate source, destination, action, severity, service, and context without source-specific code.
• Add import/export for profile templates and detector settings.
• Separate source adapters, normalizers, detectors, enrichers, and output adapters into explicit extension interfaces.
• Add optional webhook/SIEM ticket output for confirmed high-severity incidents.
• Add role-aware configuration only if the dashboard is moved beyond single-user local operation.

Acceptance: adding a new Graylog stream normally requires a profile template and configuration, not a core-code change.

Suggested Next Milestone

Complete Detection Quality first: confidence scoring, burst/rare-value detectors, per-stream thresholds, and historical replay. These features determine whether the incident and dashboard work remains trustworthy as more log sources are added.