dailyai.report

Subjective judgment tasks often fail as AI benchmarks due to inherent disagreement. A new proposal suggests measuring capabilities by asking models to predict a specific person's judgment instead. This approach aims to reduce sloppiness in conceptual reasoning. It shifts the focus from finding a "correct" answer to accurate human emulation.