Open evaluation · September 12, 2026
AI Brand Matching: 40-Case Evaluation
VisiScan's deterministic entity matcher scored 90% on 40 hand-labelled synthetic cases, with all remaining errors involving ordinary-word brand names.
Result
The deterministic matcher classified 36 of 40 hand-labelled synthetic cases correctly: 20 true positives, 16 true negatives, four false positives and zero false negatives.
Confusion matrix
| Outcome | Cases |
|---|---|
| True positive | 20 |
| True negative | 16 |
| False positive | 4 |
| False negative | 0 |
What failed
The remaining false positives are semantic ambiguities: “Apex” in “Apex Legends,” and the ordinary words “sage,” “bloom,” and “local.” Tightening a substring rule cannot reliably resolve those cases without rejecting legitimate lowercase brand mentions. They require grounded entity context or a semantic evaluator.
A shared domain parser also incorrectly rejected a bare hostname followed by punctuation. The evaluation exposed that regression; the parser now removes terminal sentence punctuation before comparing the exact hostname.
Limits
The cases are hand-authored and labelled, not sampled from customer answers. They cover exact, fuzzy, domain-only, absent, competitor-only, substring, generic-word and URL-echo cases. They do not cover the required 200-case multilingual, double-reviewed evaluator acceptance set.