Who judges the judge?
At Trusq an AI drafts answers to customer questions. Before anything is sent, a second model, the judge, checks every claim in the draft against the customer's knowledge base and removes what the knowledge base does not back. When we compared models for both jobs in September, the judge turned out to matter more than the writer.
The worst thing our product can do is not a slow reply or an incomplete one. It is a false or misleading answer, written with full confidence, that reaches a customer. The customer cannot tell it from a correct answer, and the business has to stand behind it. So we do our utmost to keep that from happening. The judge is one of the safeguards, and every quote in a reply is also checked word for word against the source article. A reply that does not pass goes to a human instead of the customer.
How we measured
We built a benchmark of 245 support questions with known correct answers, spread over two real knowledge bases: a small one with 7 articles and a larger one with 42. Every question passed a check before it counted: each fact we expected in the answer had to appear in the article text, and no phrase we counted as wrong could appear there. Then we ran a first full round and read every failure. In 15 cases the test itself was wrong, because a correct answer used other words, so we fixed those cases before freezing the set.
We ran seven writing models over the 245 questions, most of them three times, because the results vary from run to run by up to four questions per knowledge base.
What the judge did
With our setup at the time, 33 answers failed, and 16 of those failures were not the writer's fault. The judge had deleted a true sentence that the knowledge base clearly backed, such as a fee or an automatic renewal. The customer got a polite reply without the answer to the question.
We then paired the three best writers with different judges. Changing only the judge lifted each of them by 6 to 12 of the 245 questions. The gap between the three writers themselves stayed within the run-to-run noise. The best pair answered 229 of the 245 questions, against 212 for the setup we started with, and it answered in 7 to 9 seconds per ticket instead of 15 to 19.
The other direction
A judge can also be too lenient. We tested the finalists in a blind round: twelve made-up customers per knowledge base, from a neighbour to a spammer to a data-protection officer, wrote in by email and replied up to three times. A separate model graded the conversations and saw only the articles and the transcripts.
A lenient judge paired with a writer that likes to embellish let through the most unsupported statements of the round, including a promise nobody at the business would keep. That is exactly the worst case from the start of this article. The best result came from a writer that does not embellish, checked by a judge that does not over-cut. That pair misled none of the 24 customers, and it invented nothing in four benchmark runs.
What I take from it
For any pipeline in which one model checks another, measure the judge as a component of its own, and in both directions. How often does it cut something true, and how often does it let something false through? A benchmark with fixed answers shows the first. Only conversations with customers who know nothing about the business show the second.
The two failures do not weigh the same. A cut sentence costs the customer a follow-up question. A false sentence costs the business its customer's trust. We fix the first, but we guard against the second first.
If you only score the final answer, you cannot tell which of the two you are fixing.