Imagine a world where writing software tests is as simple as asking an AI. Researchers are exploring this exciting possibility by harnessing the power of large language models (LLMs), like those behind ChatGPT, to automatically generate test cases. But how good are these AI-generated tests? A new benchmark called TESTEVAL aims to find out. TESTEVAL throws a variety of Python programming problems at sixteen different LLMs, both commercial and open-source, and evaluates their performance in creating effective tests. The benchmark presents three key challenges: generating tests for overall code coverage, targeting specific lines or branches of code, and covering specific execution paths. The results are revealing. While LLMs excel at creating tests that achieve broad coverage, they struggle when asked to target specific parts of the code, suggesting a limited ability to truly understand program logic. Think of it like this: LLMs can write tests that generally poke and prod the software, but they're not yet sophisticated enough to surgically test specific functions or behaviors. Interestingly, commercial LLMs, such as GPT-4, generally outperformed open-source models. This suggests that increased resources and training data can boost performance in test generation. The TESTEVAL benchmark provides a valuable tool for researchers to evaluate and refine LLM-based testing methods. It highlights the current limitations of LLMs in this domain, while also pointing towards a future where AI plays a larger role in ensuring software quality. The challenge now is to improve the reasoning capabilities of LLMs so they can create tests as effectively as seasoned software engineers, paving the way for faster, more reliable software development.