The rise of large language models (LLMs) has fueled an explosion of AI chatbots. But how do we know which ones are truly the best? Existing ranking systems, like the Elo rating used in platforms like Chatbot Arena, often fall short. They struggle to handle ties – a frequent occurrence in human evaluations – and fail to capture the nuanced relationships between different chatbots. This research introduces a new statistical framework that tackles these limitations head-on. The key innovation is a 'factored tie model.' Think of it as uncovering the hidden patterns in how and why judges declare ties between chatbots. This model significantly improves accuracy, reducing errors in predicting ties by a remarkable two orders of magnitude! It also enhances the prediction of wins and losses. Beyond ties, the framework also models the 'covariance' between competitors. This reveals correlations in performance, providing insights beyond simple rankings. For example, it allows grouping similar chatbots into performance tiers, highlighting shared strengths and weaknesses. This framework doesn't just rank—it reveals hidden connections and offers a deeper understanding of how these AI chatbots stack up against each other. To make this powerful tool accessible, the researchers released an open-source Python package called 'leaderbot.' This package empowers anyone to reproduce these analyses, apply the framework to new chatbot data, and contribute to the ongoing evolution of AI chatbot evaluation.