dailyai.report

A new exploratory project uses black-box LLM autoraters to identify 10-20 qualitative features within model transcripts. By splitting data into user turns, thoughts, and responses, researchers can uncover surprising correlations in deployment and RL training. This automated feature discovery helps safety researchers find novel behaviors without manual auditing. It streamlines the qualitative analysis of complex model distributions.