Annotator agreement and LLM judges versus human judges

Explain annotator agreement rates (such as Cohen's kappa) and their limits. Compare human raters with an LLM as a judge: cost, consistency, bias and where each fails. Then say how you would make an LLM judge more reliable and check it against people.