Let the dashboard LLM note read a bounded excerpt of real log content

The Ollama dashboard assessment was built entirely from pre-computed
statistical scores/reasons/counts — a deliberately tested compaction step
stripped all raw message content before it reached the model, presumably to
keep the prompt small for a local, GPU-constrained instance. That raw
content (sample_events/timeline, already truncated to short messages)
already flows through the pipeline; it was just filtered out at the last
step. Now the top-scoring field deviations and incidents each carry a
small, bounded number of real message excerpts, and the prompt explicitly
asks the model to flag suspicious content (injection attempts, unusual
command lines) that pure statistical scoring can't see — purely a
qualitative addition to the narrative note, never feeding back into scoring
or suppression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-24 07:04:58 +02:00
parent caa37de98e
commit b0cc9dbdfe
4 changed files with 54 additions and 1 deletions

View File

@@ -93,6 +93,9 @@ class LlmTests(unittest.TestCase):
self.assertEqual(compact["field_deviations"][0]["entity"], "alice")
self.assertNotIn("sample_events", compact["field_deviations"][0])
self.assertNotIn("graylog_query", str(compact))
# A short, bounded message excerpt is deliberately kept so the LLM can
# reason about actual content, not only the statistical score.
self.assertEqual(compact["field_deviations"][0]["sample_messages"], ["very large raw event"])
def test_dashboard_compaction_strips_incident_and_correlation_samples(self):
status = {
@@ -132,7 +135,10 @@ class LlmTests(unittest.TestCase):
self.assertNotIn("samples", compact_text)
self.assertNotIn("sample_events", compact_text)
self.assertNotIn("graylog_query", compact_text)
self.assertNotIn("raw incident evidence", compact_text)
# Correlation samples and the raw anomaly evidence payload stay stripped
# entirely (no real content exists there today), but an incident's own
# timeline contributes a few bounded message excerpts.
self.assertEqual(compact["incidents"][0]["sample_messages"], ["raw incident evidence"])
self.assertEqual(compact["incidents"][0]["entity"], "10.0.0.5")