Large language models (LLMs) have revolutionized how we interact with technology, but they're not without their quirks. One peculiar issue? LLMs often exhibit an odd preference for certain multiple-choice answers, picking the last option or a specific letter, regardless of correctness. This "selection bias" can be a real problem for applications like automated testing and data annotation. New research tackles this head-on, exploring why LLMs develop these biases and how to fix them. One clever approach, called Bias Node Pruning (BNP), targets the LLM’s internal structure. By removing specific components of the model, BNP snips away the source of bias, like a surgeon removing a troublesome bit of code. For cases where you can't tinker with the model's internals, there's a simpler trick: Auxiliary Option Injection (AOI). This involves adding an “I don’t know” option. This seemingly trivial change can make a big difference, prompting the LLM to think twice before defaulting to a biased choice. To accurately measure selection bias, researchers have developed a new metric called Choice Kullback-Leibler Divergence (CKLD). This metric assesses how much the AI’s answer distribution deviates from the actual distribution of correct answers, providing a more reliable bias measurement than older methods. Testing these techniques on various LLMs and datasets yielded some pretty impressive results. Accuracy improved dramatically, sometimes by as much as 25%, simply by pruning a few nodes or adding an “I don’t know” choice. Notably, these methods can also boost other LLM techniques like Chain-of-Thought and In-Context Learning. These findings have big implications for making LLMs more reliable. From improving educational tools to ensuring fairer AI-driven assessments, debiasing LLMs is crucial for building trust in these powerful tools. While this research offers effective ways to mitigate selection bias, the ultimate cause remains a mystery. Future research will delve into the roots of this behavior, uncovering the underlying mechanisms that cause LLMs to develop these quirks in the first place.