Large language models (LLMs) have become the go-to for powerful text embeddings, but their massive size creates a bottleneck for speed and memory. Recent research suggests these giants might be carrying more weight than they need. By strategically pruning the last few layers of an LLM before the standard supervised contrastive training, researchers have discovered a surprising efficiency boost. Imagine shrinking these models by up to 30% *without* sacrificing performance—and up to a whopping 80% with only a modest dip in accuracy. This simple technique, requiring a mere three lines of code, challenges the assumption that bigger is always better in the world of AI. It points to a future where leaner, faster LLMs can still deliver exceptional text embeddings, making them more accessible for practical applications. The secret lies in recognizing that for text encoding, the deep layers of these models might not be adding as much value as we thought. This intriguing finding opens doors for even more aggressive optimization strategies, like L[3]Prune, which pinpoints exactly which layers to trim based on initial model loss. This allows developers to fine-tune models for peak efficiency with minimal experimentation. The study underscores that while LLMs have revolutionized NLP, there's still room for clever optimizations that could bring the power of these models to even wider audiences.