Imagine a world where machines judge what's relevant for us. Tempting, right? A recent research paper argues against using Large Language Models (LLMs) like ChatGPT to determine relevance, especially for evaluating search systems. The core issue? LLMs, when used this way, create a relevance ceiling. They become the ultimate judges, making it impossible to measure any system that might surpass their capabilities. If a new, potentially better search engine finds something the LLM deemed irrelevant, it gets penalized. This creates a paradox where advancements are punished rather than rewarded. Moreover, using an LLM for both relevance judgment and as a component of the search system itself creates a feedback loop, essentially trapping progress within the LLM's limitations. The better systems become at retrieving information that the LLM didn't deem relevant, the "worse" they appear. The future of search should be about systems that go above and beyond current capabilities, and using LLMs to make relevance judgments directly hinders this progress. The paper does offer a glimmer of hope: LLMs could be used to assist human assessors, not replace them, by acting as sophisticated quality control checkers to catch human error. They could potentially even automate parts of user studies, helping make experiments larger and more insightful. The key takeaway? LLMs have tremendous potential in search, but using them to define relevance stifles innovation and traps progress within their own limitations. Let's explore how they can support, not replace, the complex task of relevance assessment in the future.