Citing sources correctly is crucial for any writer, especially in academic and professional settings. But can large language models (LLMs), designed to generate human-like text, reliably handle this essential task? A new research paper and benchmark called REASONS (Retrieval and Automated CitationS Of Scientific Sentences) puts LLMs to the test, revealing some surprising results. The study explored how well different LLMs, including popular ones like GPT-3.5 and GPT-4, could accurately cite scientific papers. They used two main ways of asking the LLMs: "direct queries," where they asked for the authors of a given paper, and "indirect queries," where they gave the LLM a sentence from one paper and asked it to identify the paper it was citing. The researchers built a massive dataset of scientific papers from various fields like computer vision, robotics, and AI, to see how the LLMs performed across different subjects. What they found was a mixed bag. While some LLMs did reasonably well, especially with direct queries, many struggled, particularly with indirect queries. Often, they would either "pass" on the question entirely or, worse, hallucinate citations, making up references that didn't exist. Interestingly, even LLMs specifically designed for citation generation, like Perplexity.ai, sometimes fell short. The research also tested a technique called Retrieval Augmented Generation (RAG), which allows LLMs to access external information to improve their responses. They found that RAG significantly boosted the performance of some LLMs, making them more accurate and less prone to hallucinations. However, even with RAG, some LLMs still struggled with more complex or nuanced queries. One key takeaway is that simply having access to more information isn't enough. The LLMs need to understand the context and meaning of the text to cite sources correctly. This is where many of them currently fall short. The REASONS benchmark provides a valuable tool for evaluating the citation capabilities of LLMs and highlights the need for further research in this area. As LLMs become more integrated into our lives, ensuring they can accurately and reliably cite sources is crucial for maintaining trust and preventing the spread of misinformation.