A new exploratory project uses a black box LLM autorater to identify 10-20 specific features within model transcripts. Researchers split data into user turns, thoughts, and responses to uncover novel behaviors. This method helps AI Alignment Forum contributors find surprising correlations. Practitioners can now qualitatively analyze target model distributions more efficiently.