Large language models (LLMs) are massive, often containing billions of parameters. This sheer size presents significant challenges for storing and transmitting these complex models efficiently. While techniques like pruning, quantization, and decomposition have made strides in compressing these models, they often overlook a crucial aspect: inherent symmetries within the model weights. Think of it like a beautifully symmetrical snowflake—encoding every intricate detail is redundant when you can describe its underlying structure and replicate the rest. Similarly, LLMs contain rotational symmetries within their weight matrices. Encoding these weights without acknowledging these symmetries wastes precious bits on redundant information. This research introduces a clever method based on "bits-back coding" to address this redundancy. By recognizing and exploiting rotational symmetries, they achieve a 3-5% reduction in total bit usage for pruned LLMs without impacting model performance. It's like getting free storage space simply by being more organized! How does it work? The technique involves strategically rotating weight matrices to align them with their principal components. By encoding this rotated version and the instructions for the rotation itself, we avoid explicitly storing redundant information captured by the symmetry. During decoding, the original weights are reconstructed using this rotation information, effectively recovering the “snowflake” from its encoded instructions. A key challenge lies in numerical inaccuracies introduced by finite-precision representations (like float16) used in practice. To mitigate this, a small correction code is transmitted alongside the encoded weights, ensuring that any significant deviations from the original weights are addressed. This research opens up an exciting new avenue for model compression. By further refining techniques for encoding and decoding these rotations and integrating them with existing compression methods, we could see even more substantial savings in the future. This would make deploying and sharing these powerful AI models more accessible, paving the way for broader adoption across various applications.