larssand caa37de98e Fix cardinality judged against sample size instead of real event volume ...

classify_field_shape() received a deduplicated set of observed values with
no real event-volume context, so cardinality was always measured against
itself — trivially 100%, making any field with more than ~16 distinct
values look "high cardinality" no matter how large the real traffic volume
was. This misclassified genuinely low-cardinality fields (e.g. username: 82
distinct values across 1,601 events) as free text, which the new shape-veto
work then surfaced as false-positive "remove this field" warnings in
production. Fixed by passing real event coverage as an explicit
total_events parameter; omitting it keeps prior behavior for other callers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-23 22:02:44 +02:00

5.2 KiB

Raw Blame History

View Raw