Large language models (LLMs) are impressive, but they're not perfect. One major flaw? Bias. LLMs can be swayed by seemingly insignificant details like word choice, example order, or even formatting, leading to inconsistent and sometimes unfair results. This "prompt brittleness" has puzzled researchers, who have tried to fix it by tweaking the outputs. But what if the problem lies deeper, within the AI's internal workings? That's the question researchers tackled in a new paper called "UniBias." Instead of just patching the outputs, they dug into the LLM's core components—the attention heads and feedforward networks (FFNs). Think of attention heads as the AI's focus mechanism, deciding which parts of a text to prioritize. FFNs, on the other hand, process and transform information. The researchers found that certain attention heads and FFN vectors consistently favor specific labels, regardless of the input. Imagine an LLM tasked with sentiment analysis. A biased attention head might fixate on the last word of a sentence, while a biased FFN might inherently lean towards "positive" regardless of the actual sentiment. This internal bias explains why LLMs are so sensitive to prompt design—the biases amplify small variations, leading to unpredictable swings in accuracy. To combat this, the researchers developed UniBias, a method that identifies and neutralizes these biased components. By essentially masking these troublemakers, UniBias allows the LLM to make decisions based on the actual content, not its internal biases. The results are impressive. Across 12 different datasets, UniBias significantly boosted accuracy and, crucially, reduced prompt brittleness. This means more consistent and reliable results, regardless of how the prompt is phrased. The implications are huge. By understanding and mitigating internal biases, we can build more robust and fair AI systems. UniBias is a step towards more transparent and trustworthy LLMs, paving the way for AI that's not just smart, but also equitable.