Public/fgAI
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix view in profile stream field and readme

Browse Source
This commit is contained in:
larssand
2026-06-22 21:10:17 +02:00
parent c1550585d9
commit 9c51b92303
2 changed files with 47 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
README.md
View File

@@ -27,6 +27,42 @@ Or use the helper script, which creates/uses `.venv` automatically and runs `pip
- Continuous monitor writing `state/fgai-status.json`
- Local dashboard at `http://127.0.0.1:8088`
## Graylog MCP
Graylog 7.1 MCP can be used as the active log source instead of the local JSONL
listener. In the dashboard, open `Settings`, select `Graylog MCP`, provide the
MCP URL and a read-only API token, then load and enable the streams to analyze.
The token field accepts a raw Graylog API token, the Base64 value after `Basic `,
or a complete `Basic <value>` header. Tokens are stored only in the local runtime
configuration and are never returned by the dashboard API.
Use `Load selected stream fields` after choosing a stream. The field table shows
Graylog datatype/capability metadata and lets you select an entity field, a time
field, and categorical/numeric fields for the stream profile. Profiles are stored
under `graylog_stream_profiles` in `state/fgai-config.json`.
Enabled streams are normalized through the same event model. The dashboard and
Ollama correlate source IPs that occur across two or more streams, for example
FortiGate, AdGuard/DNS, Windows Security, Nginx, Squid, VPN, or Proxmox.
The current MCP endpoint is `http://<graylog-host>:9000/api/mcp`. Enable it in
Graylog under `System -> Configurations -> MCP` and use stream IDs internally;
the fgAI stream picker resolves titles in the UI.
## Monitoring Export
The dashboard also exposes Prometheus text metrics at:
```text
http://127.0.0.1:8088/metrics
```
This endpoint is passive and has no Prometheus or Grafana dependency. It reports
low-cardinality event counts, anomaly severities, baseline readiness, and Graylog
MCP health. Use it later as a Prometheus scrape target or as input for a Checkmk
local check. Do not use source IPs, domains, or raw event IDs as metric labels.
Enable cached Ollama analyst notes in the dashboard:
```bash
@@ -200,7 +236,7 @@ end
- `FORTIGATE_VERIFY_TLS`: `true` or `false`, defaults to `true`.
- `FGAI_ALLOWLIST`: comma-separated IPs/CIDRs never to block.
- `OLLAMA_HOST`: defaults to `http://127.0.0.1:11434`.
- `OLLAMA_MODEL`: defaults to `llama3.3`.
- `OLLAMA_MODEL`: defaults to `llama3.1`.
- `OLLAMA_TIMEOUT`: Ollama request timeout in seconds, defaults to `180`.
- `FGAI_LLM`: set to `1` to enable dashboard Ollama analyst notes.
- `FGAI_LLM_INTERVAL`: seconds between dashboard LLM notes, defaults to `300`.

11
src/fgai/dashboard.py
View File

@@ -47,6 +47,14 @@ HTML = """<!doctype html>
[data-view] { display: none; } [data-view].active { display: block; }
.split { display: grid; grid-template-columns: minmax(0, 1.3fr) minmax(320px, 0.7fr); gap: 12px; }
.table-wrap { overflow-x: auto; }
#settingsForm label:has(#fieldPicker) { grid-column: 1 / -1; }
#fieldPicker { margin-top: 8px; max-height: 460px; overflow: auto; border: 1px solid #d9e0e7; background: #fbfcfd; }
.field-header, .field-row { display: grid; grid-template-columns: minmax(190px, 1.2fr) 100px minmax(180px, 1fr) minmax(320px, 1.4fr); gap: 10px; align-items: center; padding: 8px 10px; }
.field-header { position: sticky; top: 0; background: #edf3f7; color: #536170; font-size: 12px; font-weight: 700; z-index: 1; }
.field-row { border-top: 1px solid #e4e9ee; font-size: 13px; }
.field-row code { width: fit-content; }
.field-controls { display: flex; flex-wrap: wrap; gap: 10px; }
.field-controls label { white-space: nowrap; }
@media (max-width: 860px) { .hero, .split { grid-template-columns: 1fr; } .hero img { display: none; } }
</style>
</head>
@@ -199,7 +207,8 @@ document.getElementById('loadFields').addEventListener('click', async () => {
if (!selected) { document.getElementById('fieldPicker').textContent = 'Load streams, tick one stream, then load its fields.'; return; }
const payload = await (await fetch(`/api/graylog/fields?stream_id=${encodeURIComponent(selected.dataset.id)}`)).json();
const profile = (window.streamProfiles || []).find(item => item.stream_id === selected.dataset.id) || {};
document.getElementById('fieldPicker').innerHTML = (payload.fields || []).map(field => { const name=field.name||field.field, props=(field.type||{}).properties||[]; return `<div><code>${esc(name)}</code> <label>entity <input type="radio" name="profile_entity" value="${esc(name)}" ${profile.entity_field===name?'checked':''}></label> <label>time <input type="radio" name="profile_timestamp" value="${esc(name)}" ${profile.timestamp_field===name?'checked':''}></label> ${props.includes('enumerable')?`<label>categorical <input class="profile-categorical" type="checkbox" value="${esc(name)}" ${(profile.categorical_fields||[]).includes(name)?'checked':''}></label>`:''} ${props.includes('numeric')?`<label>numeric <input class="profile-numeric" type="checkbox" value="${esc(name)}" ${(profile.numeric_fields||[]).includes(name)?'checked':''}></label>`:''}</div>`; }).join('') || esc(payload.error || 'No fields found.');
const rows = (payload.fields || []).map(field => { const name=field.name||field.field, type=(field.type||{}).type||'', props=(field.type||{}).properties||[]; return `<div class="field-row"><code>${esc(name)}</code><span>${esc(type)}</span><span>${esc(props.join(', '))}</span><div class="field-controls"><label><input type="radio" name="profile_entity" value="${esc(name)}" ${profile.entity_field===name?'checked':''}> Entity</label><label><input type="radio" name="profile_timestamp" value="${esc(name)}" ${profile.timestamp_field===name?'checked':''}> Time</label>${props.includes('enumerable')?`<label><input class="profile-categorical" type="checkbox" value="${esc(name)}" ${(profile.categorical_fields||[]).includes(name)?'checked':''}> Categorical</label>`:''}${props.includes('numeric')?`<label><input class="profile-numeric" type="checkbox" value="${esc(name)}" ${(profile.numeric_fields||[]).includes(name)?'checked':''}> Numeric</label>`:''}</div></div>`; }).join('');
document.getElementById('fieldPicker').innerHTML = rows ? `<div class="field-header"><span>Field</span><span>Type</span><span>Capabilities</span><span>Use In Profile</span></div>${rows}` : esc(payload.error || 'No fields found.');
window.activeProfileStream = selected.dataset.id;
});
document.getElementById('settingsForm').addEventListener('submit', async event => {