Reverb docs
Judge Validation
Why the report includes a trust badge instead of treating LLM judgment as unquestioned truth.
1
Each answer is judged against a small fixed schema: target mention, prominence, competitors mentioned, and cited domains.
2
Brand and product aliases are normalized per scorecard before trusting mention counts.
3
The validation queue samples about 40 answer judgments, or all answers when fewer are available.
4
A human reviewer labels target mention, prominence, and competitor mentions for the sampled answers.
5
The public report shows validation pending, trusted, or needs review based on field-level agreement.
6
The default trust threshold is 85% overall agreement across labeled fields.