# SignalScope Roadmap

This roadmap tracks the path from the current local Graylog analytics agent to a
production-ready, multi-source security analytics tool. Items are ordered by
operational value and dependency, not by UI appeal.

## Current Foundation

- [x] Graylog MCP connection, stream discovery, field discovery, and stream selection.
- [x] Per-stream profiles for entity, timestamp, categorical, and numeric fields.
- [x] Local five-minute field baselines with duplicate-event protection.
- [x] Time-aware baseline comparison using matching UTC weekday/hour when available.
- [x] Generic entity correlation for IP addresses, users, and hostnames.
- [x] Cross-stream correlation timelines and investigation incident grouping.
- [x] Field-deviation review: expected, false positive, confirmed, note, and expiry.
- [x] Local Ollama analyst assessment with incident and feedback context.
- [x] Cached AbuseIPDB/VirusTotal enrichment with a daily request budget.
- [x] Prometheus-compatible low-cardinality metrics endpoint.
- [x] Local dashboard for status, findings, incidents, diagnostics, and configuration.

## 1. Detection Quality

Goal: make findings more accurate before adding more integrations.

- [x] Add baseline confidence based on sample count and time-bucket coverage.
- [x] Add generic event-rate burst detection per stream/entity.
- [x] Add rare-value detection with a minimum historical observation threshold.
- [x] Add detector-specific authentication failure, DNS volume, and denied-traffic burst thresholds.
- [x] Add configurable per-field detector weights.
- [x] Add sequence detection, for example DNS lookup -> outbound connection -> authentication event.
- [x] Add per-stream detector enablement and thresholds in the UI.
- [x] Add a dry-run replay command for historic JSONL or Graylog exports using temporary baselines.
- [x] Add direct Graylog MCP time-range replay and result comparison against saved detector configurations.
- [ ] Add dashboard controls for launching safe replay jobs and viewing detector deltas.

Acceptance: each finding shows its detector, confidence, baseline sample count, current value, expected value, and a bounded set of raw-event references.

## 2. Entity and Incident Investigation

Goal: make one incident answer what happened, to whom, and across which sources.

- [ ] Allow multiple entity fields per stream, such as user plus source IP plus hostname.
- [ ] Add entity aliasing: map DHCP, VPN, DNS, and endpoint identities to the same host where evidence supports it.
- [ ] Add configurable incident grouping windows and incident lifecycle: open, acknowledged, resolved, reopened.
- [ ] Persist incident state and analyst notes separately from transient detection output.
- [ ] Add direct Graylog query links or query details for each timeline event.
- [ ] Add investigation export as JSON and Markdown report.

Acceptance: an analyst can open an incident, see an ordered multi-stream timeline, review evidence, and record an outcome without losing it after the next monitor poll.

## 3. Feedback and Learning Controls

Goal: reduce repeat noise without hiding genuinely new behavior.

- [ ] Show active feedback rules, expiry, scope, author, and matching findings in the UI.
- [ ] Support feedback scope choices: exact value, entity and field, entire entity, or stream-wide rule.
- [ ] Require an expiry for broad suppressions and warn before a broad rule is saved.
- [ ] Track detector precision using confirmed versus false-positive feedback.
- [ ] Include feedback-derived precision and suppression counts in metrics and diagnostics.

Acceptance: a false-positive decision suppresses only the intended pattern and all suppressions are auditable and reversible.

## 4. Threat Intelligence and Enrichment

Goal: enrich public indicators predictably without exhausting provider quotas.

- [ ] Make cache TTL and daily budget configurable in the dashboard.
- [ ] Add provider health, cache hit ratio, and quota/budget visibility.
- [ ] Add domain and URL enrichment behind separate provider budgets.
- [ ] Keep enrichment asynchronous so a slow provider never delays monitoring.
- [ ] Store enrichment provenance and observation timestamps with incident evidence.

Acceptance: no refresh cycle causes uncontrolled external lookups, and every reputation result shows provider, timestamp, and cache state.

## 5. Production Operations

Goal: run reliably in a monitored environment.

- [ ] Add systemd unit files for monitor, dashboard, and optional local syslog listener.
- [ ] Add health and readiness endpoints with last successful Graylog fetch time.
- [ ] Add structured application logs and configurable retention for status/history/baseline data.
- [ ] Add backup and migration procedure for SQLite state.
- [ ] Add Checkmk local-check output in addition to Prometheus metrics.
- [ ] Add authentication/reverse-proxy guidance before exposing the dashboard beyond loopback.

Acceptance: deployment, restart, upgrade, backup, and monitoring have documented, repeatable procedures.

## 6. Dashboard Usability

Goal: optimize the UI for security investigation rather than raw tables.

- [x] Separate stream enablement from one-profile-at-a-time profile editing.
- [x] Add stream coverage diagnostics for enabled/profiled/ready/no-event streams.
- [ ] Replace remaining long tables with compact incident and entity cards where appropriate.
- [ ] Add incident filters for stream, severity, entity type, review state, and time range.
- [ ] Add baseline versus current charts per selected entity and field.
- [ ] Add an interactive entity graph backed by actual correlations.
- [ ] Preserve user view state, filters, expanded evidence, and selected tab across refreshes.
- [ ] Add visual data-quality warnings for missing timestamps, entity fields, or truncated Graylog results.

Acceptance: common triage can be completed from the dashboard without manually parsing raw JSON or searching for stream IDs.

## 7. Integrations and Extensibility

Goal: add log sources and outputs without adding source-specific logic everywhere.

- [ ] Define versioned stream-profile templates for FortiGate, Windows, DNS/AdGuard, Nginx, Squid, VPN, and Proxmox.
- [x] Add inventory-style stream coverage to guide which streams need profiles before templates are added.
- [ ] Add import/export for profile templates and detector settings.
- [ ] Separate source adapters, normalizers, detectors, enrichers, and output adapters into explicit extension interfaces.
- [ ] Add optional webhook/SIEM ticket output for confirmed high-severity incidents.
- [ ] Add role-aware configuration only if the dashboard is moved beyond single-user local operation.

Acceptance: adding a new Graylog stream normally requires a profile template and configuration, not a core-code change.

## Suggested Next Milestone

Complete Detection Quality first: confidence scoring, burst/rare-value detectors,
per-stream thresholds, and historical replay. These features determine whether
the incident and dashboard work remains trustworthy as more log sources are added.