Large language models (LLMs) have revolutionized how we represent text, powering advanced embedding models trained on massive datasets covering diverse topics. These powerful general-purpose embeddings achieve impressive results on benchmarks like MTEB. But a crucial question emerges: are these generalist models sufficient, or do we still need specialized embeddings for specific fields? This research delves into this question, focusing on the complex world of finance. Researchers created a new benchmark, FinMTEB, mirroring MTEB but using datasets exclusively from the financial domain. Testing several state-of-the-art embedding models, they found a significant performance drop on FinMTEB tasks. To ensure this wasn't simply due to FinMTEB's complexity, they used four measures—ChatGPT error rates, readability, information entropy, and dependency distance—to level the playing field between the benchmarks. Even after controlling for complexity, the performance gap remained, highlighting the difficulty general-purpose models face when grappling with domain-specific nuances. Notably, models excelling on MTEB didn't necessarily shine on FinMTEB, emphasizing the need for specialized benchmarks to evaluate domain-specific performance. This research strongly suggests that crafting domain-specific embeddings remains essential, even in the LLM era. However, the question of *how* to create these specialized models—adapting LLMs or fine-tuning general models with domain-specific data—remains an active research area. The introduction of FinMTEB marks a vital step toward more refined benchmarks, providing a valuable platform for future research and development in financial embeddings, and pushing AI's boundaries in handling specialized fields.