In today's AI-driven world, we rely heavily on search engines and question-answering systems to access information quickly and efficiently. But how can we ensure that these systems provide accurate and comprehensive answers, especially when dealing with complex or open-ended questions? Researchers have developed a new method called Retrieval-Augmented Generative Question Answering (RAG-QA), where AI models retrieve relevant information from a vast knowledge base to answer questions. However, evaluating the *robustness* of these RAG-QA systems across different domains has been a challenge. Existing datasets often focus on short, extractive answers or single-source corpora, limiting their ability to assess the true capabilities of these systems. Enter the *RAG-QA Arena*, a novel evaluation platform designed to test the mettle of even the most sophisticated AI models. The Arena builds upon a new dataset called Long-form RobustQA (LFRQA), containing thousands of human-written long-form answers that weave together information from multiple documents across seven diverse domains. These answers are not simply concatenated snippets but carefully crafted narratives that reconcile potentially conflicting information. The RAG-QA Arena pits leading LLMs against LFRQA in a head-to-head comparison, judged by both humans and other LLMs acting as evaluators. The results are intriguing. Even the most advanced LLMs often struggle to match the quality and completeness of LFRQA's human-crafted answers, especially when dealing with questions requiring synthesis of information from various sources. The RAG-QA Arena offers a more realistic and challenging benchmark for evaluating long-form question answering systems. By comparing AI-generated answers against carefully curated human answers, the Arena reveals the strengths and weaknesses of current AI models and highlights opportunities for improvement. This research not only unveils the limitations of current technology but also paves the way for developing more robust and reliable AI question-answering systems across diverse fields.