Large language models (LLMs) are transforming how we interact with technology, but their potential for bias poses a significant challenge. While researchers have developed numerous tools and metrics to measure these representational harms—instances where AI systems portray certain social groups less favorably—a critical gap exists between these academic efforts and the practical needs of developers building real-world LLM applications. A new study reveals the key obstacles preventing practitioners from effectively using existing bias-measuring tools. Through interviews with AI practitioners across various roles and organizations, researchers uncovered four main categories of challenges. First, existing tools often lack validity, meaning they don't accurately measure what they claim to, or they rely on contested definitions of harm. Furthermore, many tools aren't specific enough to apply to particular LLM systems, use cases, and deployment contexts. Generic datasets and broad labels like "hate speech" don't provide the granular insights developers need. Second, practical constraints in real-world development pose significant hurdles. Practitioners often prioritize software testing practices for quality assurance, deal with data licensing and security issues, and struggle to find the time to evaluate existing tools, ultimately leading them to create new, internal measurement instruments instead. Third, the opaque nature of LLM training data complicates benchmark usage. Developers are wary of using public benchmarks, fearing their systems may simply be replicating patterns from the very data used to evaluate them. This opacity necessitates creating custom internal benchmarks, adding to the complexity. Finally, measuring representational harm presents unique challenges compared to other types of AI harms. Practitioners emphasize the need for contextual understanding, agreement on complex social constructs, and social science expertise, resources that aren't always readily available. Furthermore, they report less commercial pressure to address representational harms compared to issues like service quality. This research highlights the urgent need to bridge the gap between academic research and practical application in measuring and mitigating LLM bias. Future work must focus on developing more valid, specific, and practical tools while also fostering better communication and collaboration between researchers and practitioners. Only then can we hope to unlock the full potential of LLMs while mitigating their potential for social harm.