A new exploratory project uses black box LLMs to identify 10-20 distinct features within model transcripts. By splitting data into user turns, thoughts, and responses, researchers can qualitatively map target model behaviors. This method helps identify surprising correlations in deployment and RL training. Practitioners can now automate the discovery of novel behaviors in complex distributions.