Safety
50d agoNLAs Fail To Detect Implausible Training Data
A 99.3% rate of implausible statements occurred when Qwen2.5-7B natural language autoencoders were initialized with nonsense data. Despite this, reconstruction accuracy remained nearly identical to plausible models. This suggests Claude-generated guesses may mislead interpretability tools without impacting performance.
The Signal
Practitioners cannot rely on NLA text outputs to verify internal model states.