Imagine having a brilliant student who excels at nearly everything, yet struggles with simple instructions. That's the current state of Large Language Models (LLMs) when applied to search ranking. These powerful AIs can generate human-quality text, translate languages, and even write different kinds of creative content. But when it comes to ranking search results, their performance is surprisingly sensitive to the way they are prompted. A new study investigated how different prompt variations affect the effectiveness of zero-shot LLM-based rankers, uncovering surprising insights. Researchers delved into the nuances of prompts, examining elements like task instructions, the tone of the wording, and even whether the prompt engages in "role-playing" by asking the LLM to act as a specific ranking tool. They discovered that these seemingly minor tweaks can drastically alter the quality of search rankings, often outperforming the original prompts used by researchers. For instance, instructing an LLM to judge relevance on a numerical scale often yielded poorer results, and including tone words was generally beneficial. This means what we ask an LLM and how we ask are just as crucial to the ranking algorithm itself. What makes these findings even more compelling is their impact on the stability of ranking methods. The research revealed that some ranking algorithms are extremely vulnerable to prompt variations, exhibiting significant performance fluctuations depending on the prompt's wording. Others proved more resilient, suggesting the need for greater attention to prompt optimization strategies to mitigate these vulnerabilities. Overall, this research underscores the importance of prompt engineering in maximizing LLM potential for search ranking. By crafting carefully optimized prompts, we can unlock the true power of LLMs and pave the way for more relevant and accurate search results.