Large language models (LLMs) are impressive, but are they truly unbiased? New research reveals a hidden bias: LLMs might be significantly influenced by how they’re asked to present information. The study, “LLMs Are Biased Towards Output Formats!”, systematically explored how different output formats affect LLM performance. Researchers discovered that asking an LLM to provide an answer as a bulleted list, a quoted phrase, or simply a letter (like in multiple-choice questions) can drastically alter its accuracy. This "format bias" stems from the uneven distribution of these formats in the LLMs’ training data. For example, models trained heavily on code excel when asked to give answers as Python lists but struggle with other formats. This bias isn’t just theoretical—it impacts real-world applications. Imagine an AI medical diagnosis system that's more accurate if it gives a single-letter answer than a detailed explanation. Clearly, consistency is key for reliable AI. The researchers offer promising solutions. By providing the models with demonstrations of various formats or simply repeating format instructions, they noticed a decrease in bias. More impressively, fine-tuning the LLM on data synthesized across formats almost eliminated the issue. This research highlights the importance of critically evaluating LLMs beyond just their knowledge—we also need to understand how they communicate that knowledge. Addressing format bias is vital for building truly robust, unbiased, and trustworthy AI systems across diverse applications, from healthcare to education and beyond.