We often hear about AI's amazing abilities, but what about its skills in judgment? Researchers are exploring using large language models (LLMs) to score things like written text and dialogues, essentially acting as automated judges. But it turns out, getting AI to score consistently is surprisingly tricky. A new study reveals that even small changes in how you instruct an LLM can significantly impact its scoring. Imagine a teacher grading essays – if the instructions aren’t perfectly clear, the grades might be all over the place. It's similar with LLMs. This research focused on how the *order* of instructions affects an LLM's evaluation of dialogues. Researchers experimented with telling the LLM to give a score *before* explaining its reasoning, and vice-versa. Surprisingly, the LLM scored dialogues higher when it gave the reasons *first*. This suggests that the act of explaining its reasoning influenced the final score, highlighting the sequential nature of how these models process information. The study also found that adding specific rules to the instructions, like prioritizing the *number* of issues over their severity, drastically changed the LLM’s scoring. This underscores how sensitive LLMs are to even subtle changes in wording, emphasizing the need for meticulous prompt engineering. This research has important implications for using LLMs in tasks requiring subjective judgment. Whether it's evaluating creative writing, assessing customer service interactions, or even helping with complex decision-making, understanding how prompting affects AI's 'judgment' is crucial for building reliable and fair AI systems. Future research could explore different instruction styles or even train LLMs to be less sensitive to prompt variations, bringing us closer to truly objective AI evaluation.