fgAI/README.md

# Fortigate AI ML Inspection Agent

Local FortiGate log and policy inspection agent. It parses FortiGate syslog/JSONL logs, audits FortiOS policy exports, highlights UTM events, and can quarantine malicious source IPs through the FortiGate API when explicitly enabled.

Autoblocking is dry-run by default. The tool will not block RFC1918, loopback, multicast, link-local, reserved, or allowlisted addresses unless you change the code.

## Quick Start

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

Or use the helper script, which creates/uses `.venv` automatically and runs `pip install -e .`:

```bash
./start.sh
./start.sh status
./start.sh analyze
./start.sh stop
```

`./start.sh` starts three local background processes:

- UDP syslog listener writing `logs/fg_syslog.jsonl`
- Continuous monitor writing `state/fgai-status.json`
- Local dashboard at `http://127.0.0.1:8088`

The script activates `.venv` inside the script process. If you also want your current shell prompt to show the venv, run:

```bash
source .venv/bin/activate
```

For UDP `514`, the script starts only the listener command with `sudo`:

```bash
FGAI_SYSLOG_PORT=514 ./start.sh
```

Analyze local logs:

```bash
fgai analyze-logs --logs logs/fg_syslog.jsonl
```

Open the live UI after `./start.sh`:

```bash
xdg-open http://127.0.0.1:8088
```

Score likely traffic anomalies:

```bash
fgai detect-anomalies --logs logs/fg_syslog.jsonl --min-score 35
fgai detect-anomalies --logs logs/fg_syslog.jsonl --min-score 35 --llm --llm-timeout 300
```

Generate response and policy recommendations:

```bash
fgai recommend --logs logs/fg_syslog.jsonl --min-score 35
```

Optional external reputation enrichment is disabled by default. To use VirusTotal for public source/destination IP reputation:

```bash
export FGAI_THREAT_INTEL=1
export ABUSEIPDB_API_KEY='...'
fgai recommend --logs logs/fg_syslog.jsonl --min-score 35 --threat-intel
```

VirusTotal is also supported:

```bash
export FGAI_THREAT_INTEL=1
export FGAI_THREAT_INTEL_PROVIDER=virustotal
export VIRUSTOTAL_API_KEY='...'
fgai recommend --logs logs/fg_syslog.jsonl --min-score 35 --threat-intel
```

Listen for FortiGate syslog locally:

```bash
fgai listen-syslog --port 5514 --output logs/fg_syslog.jsonl
```

Run the listener quietly in the background:

```bash
./start.sh
```

Stop the background listener:

```bash
./start.sh stop
```

UDP port `514` normally needs root privileges on Linux:

```bash
sudo .venv/bin/fgai listen-syslog --port 514 --output logs/fg_syslog.jsonl
```

Test FortiGate API access:

```bash
export FORTIGATE_HOST=192.0.2.10
export FORTIGATE_API_TOKEN='...'
export FORTIGATE_VERIFY_TLS=false
fgai test-connection
fgai fetch-policies --output exports/policies.json
```

Audit a FortiGate policy export:

```bash
fgai audit-policies --config exports/fortigate.conf
```

Or fetch policies through the FortiGate API and audit that JSON:

```bash
fgai fetch-policies --output exports/policies.json
fgai audit-policies --config exports/policies.json --llm --llm-timeout 300
```

Find block candidates without changing the firewall:

```bash
fgai suggest-blocks --logs logs/fg_syslog.jsonl
```

Execute guarded quarantine actions:

```bash
export FORTIGATE_HOST=192.0.2.10
export FORTIGATE_API_TOKEN='...'
fgai suggest-blocks --logs logs/fg_syslog.jsonl --execute --expiry-minutes 60
```

Optional local LLM summary through Ollama:

```bash
ollama pull llama3.3
fgai analyze-logs --logs logs/fg_syslog.jsonl --llm --llm-timeout 300
```

For slower machines or large models:

```bash
OLLAMA_MODEL=llama3.1 OLLAMA_TIMEOUT=300 fgai analyze-logs --logs logs/fg_syslog.jsonl --llm
```

## FortiGate Inputs

For logs, configure FortiGate syslog to write into a local file such as `logs/fg_syslog.jsonl`. The parser supports common key/value syslog lines and JSONL.

For policies, export a FortiOS config backup and pass it to `audit-policies`.

Example FortiGate syslog target, run on the FortiGate CLI and replace the server IP with this machine:

```text
config log syslogd setting
    set status enable
    set server "192.0.2.50"
    set port 5514
    set mode udp
    set format default
end
```

## Environment

- `FORTIGATE_HOST`: firewall hostname or IP.
- `FORTIGATE_API_TOKEN`: REST API token.
- `FORTIGATE_VERIFY_TLS`: `true` or `false`, defaults to `true`.
- `FGAI_ALLOWLIST`: comma-separated IPs/CIDRs never to block.
- `OLLAMA_HOST`: defaults to `http://127.0.0.1:11434`.
- `OLLAMA_MODEL`: defaults to `llama3.3`.
- `OLLAMA_TIMEOUT`: Ollama request timeout in seconds, defaults to `180`.
- `FGAI_THREAT_INTEL`: set to `1` to enable external threat intelligence lookups.
- `ABUSEIPDB_API_KEY`: AbuseIPDB API key for public IP reputation enrichment.
- `ABUSEIPDB_MAX_AGE_DAYS`: report age window for AbuseIPDB, defaults to `90`.
- `FGAI_THREAT_INTEL_PROVIDER`: `auto`, `abuseipdb`, or `virustotal`.
- `VIRUSTOTAL_API_KEY`: VirusTotal API key for public IP reputation enrichment.

## Safety Model

The agent separates detection from enforcement:

- UTM events are scored from FortiGate logs (`ips`, `virus`, `anomaly`, `ddos`, `webfilter`, `app-ctrl`, `waf`, `dns`).
- Source IPs must be globally routable and outside the allowlist.
- Blocking requires `--execute`.
- The FortiGate API call is limited to the quarantine/banned user monitor endpoint.