Safety
59d agoQuantization Effects on Gemma 3 Interpretability
Weight compression to 8-bit and 4-bit levels in Gemma 3 4B and 12B models alters their representational geometry. This analysis tests how quantization impacts feature activation and model performance.
The Signal
Practitioners should note that compressing weights changes the internal mechanics of interpretability, potentially obscuring the very features researchers aim to map.