Systematic literature reviews (SLRs) are essential for academic research, but they're incredibly time-consuming. Researchers often spend weeks, even years, meticulously curating publications to answer specific research questions. Could AI and large language models (LLMs) like ChatGPT be the key to automating this painstaking process? A recent study explored this possibility by examining how well LLMs can generate the complex Boolean queries that drive these reviews. These queries are the heart of SLRs, filtering through mountains of publications to pinpoint the most relevant studies. The researchers focused on replicating and expanding previous work, putting LLMs like ChatGPT, Mistral, and Zephyr to the test, using established datasets of medical reviews. They wanted to see not only how well these models generated effective queries, but also how consistent and reproducible the results were. While LLMs showed some promise, outperforming existing baselines in certain aspects, the research revealed some significant hurdles. For one, reproducing consistent results proved challenging. The queries generated by LLMs varied significantly, raising concerns about reliability for rigorous academic work. Another issue was the difficulty in achieving high recall – that is, ensuring the AI doesn't miss critical studies. Even the best-performing queries lagged behind human-crafted ones in this regard. The study also looked at open-source LLMs as a potentially more accessible alternative to commercial models. These models performed reasonably well, demonstrating their potential for wider use. However, they faced limitations in handling longer, more complex queries. Interestingly, the study highlighted that LLMs are more effective when provided with examples of well-structured queries, suggesting they learn and adapt based on the information they receive. Overall, the study presents a nuanced perspective on the potential of AI in literature reviews. While LLMs can assist in generating candidate queries and reducing some of the manual effort, they're not yet a replacement for human expertise. The inconsistency of results and the challenge of achieving high recall emphasize the need for further refinement and development before AI can truly automate the complex process of conducting a systematic literature review.