Large language models (LLMs) excel at generating human-like text. This ability has opened exciting avenues in other areas like creating text embeddings. These embeddings, which convert text into numerical representations, are at the heart of powerful applications such as semantic search and retrieval-augmented generation (RAG). But what's the most effective way to build these LLM-powered embeddings? New research delves into two crucial design choices: pooling and attention. Pooling determines how to condense the information from an LLM's output into a fixed-size vector, while attention controls which parts of the text the model focuses on. The study discovered that there's no one-size-fits-all answer. A combination of bidirectional attention (allowing the model to look at the text in both directions) and a trainable pooling layer works best for text similarity and information retrieval tasks. However, for clustering and classification, simpler approaches, like focusing on the last token and using causal attention (only looking at preceding text), perform better. The researchers also introduce a novel pooling strategy called "Multi-Layers Trainable Pooling." Instead of using only the final layer of the LLM's output, it pulls data from all layers, leading to statistically superior performance in similarity and retrieval tasks. These insights are vital for developers seeking to enhance embedding performance. There isn't a single perfect formula for creating LLM-based embeddings; the ideal approach varies depending on the task. However, the innovative pooling strategies discussed in the study offer powerful new tools for optimizing these powerful text representations.