Large language models (LLMs) like ChatGPT are impressive at answering questions, but what about *writing* them? Researchers explored a fascinating twist on question answering: reverse question answering (RQA), where the LLM is given an answer and has to craft the perfect question. The results revealed surprising gaps in LLM reasoning, especially when numbers are involved. While LLMs excelled at generating questions for factual answers, they struggled mightily with numerical answers. For example, given the number “488”, models often concocted complex, multi-step math problems or combined math with irrelevant facts, leading to incorrect or nonsensical questions. Interestingly, these same models could correctly answer these flawed questions when presented with them later, showing the problem isn’t simply a lack of knowledge. This inconsistency hints at a deeper issue with how LLMs reason—they're great at deducing answers from given premises but struggle with the inverse, abductive reasoning required for RQA. The research also suggests that LLMs have difficulty with rare numerical facts, often creating questions about obscure or even made-up information. This finding underscores the challenge of teaching LLMs to handle the 'long tail' of knowledge—the vast sea of less common but still important information. This research highlights key areas for improvement in LLM development. By addressing these weaknesses, we can build more consistent and reliable AI reasoning abilities for tasks like exam generation, brainstorming tools, and even scientific hypothesis generation. The challenges of reverse question answering offer a unique window into how LLMs think, revealing the complexities of building truly intelligent machines.